Nov 04 04:45:13.703460 ip-10-0-140-242 systemd[1]: Starting Kubernetes Kubelet... Nov 04 04:45:14.147794 ip-10-0-140-242 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 04:45:14.147794 ip-10-0-140-242 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 04:45:14.147794 ip-10-0-140-242 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 04:45:14.147794 ip-10-0-140-242 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 04:45:14.152471 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152389 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:14.152471 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152467 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:14.152471 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152471 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:14.152471 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152474 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:14.152471 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152477 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:14.152471 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152479 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:14.152471 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152482 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152485 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152488 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152491 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152494 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152497 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152499 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152502 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152504 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152507 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152515 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152518 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152520 2577 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152523 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152526 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152528 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152531 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152534 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152536 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152539 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:14.152744 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152541 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152544 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152547 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152549 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152552 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152554 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152557 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152559 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152562 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152564 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152567 2577 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152570 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152572 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152576 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152579 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152582 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152585 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152588 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152590 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152593 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:14.153232 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152595 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152598 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152600 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152603 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152605 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152608 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152611 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152614 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152616 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152618 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152621 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152623 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152626 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152628 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152631 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152634 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152637 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152640 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152642 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:14.153736 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152645 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152648 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152651 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152653 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152656 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152661 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152666 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152670 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152674 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152677 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152680 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152684 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152687 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152689 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152691 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152694 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152697 2577 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152699 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152702 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:14.154209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152705 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152707 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152710 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152712 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152715 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152717 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152720 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152729 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152732 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152735 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152738 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152741 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152744 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152747 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152750 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152753 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152755 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152758 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152761 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152763 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:14.154655 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152766 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.152770 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153584 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153592 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153595 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153598 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153601 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153604 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153606 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153609 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153612 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153614 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153617 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153619 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153622 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153625 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153627 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153629 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153632 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:14.155131 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153635 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153638 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153640 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153643 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153646 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153649 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153652 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153655 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153657 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153660 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153662 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153665 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153668 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153670 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153673 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153677 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153680 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153682 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153684 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153687 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:14.155592 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153689 2577 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153692 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153694 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153697 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153699 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153702 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153704 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153707 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153709 2577 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153712 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153714 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153717 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153719 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153722 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153724 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153729 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153733 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153738 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153741 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:14.156079 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153744 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153746 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153749 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153752 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153755 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153758 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153760 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153763 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153765 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153768 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153771 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153773 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153776 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153779 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153782 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153785 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153787 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153790 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153792 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153795 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:14.156548 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153797 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153800 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153804 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153808 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153811 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153813 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153817 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153820 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153824 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153826 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153829 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153831 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153834 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153836 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153839 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153841 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153844 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153846 2577 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153849 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:14.157022 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153851 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153855 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153857 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153860 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153863 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153865 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153868 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153870 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153872 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153875 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.153877 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154364 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154370 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154374 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154377 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154380 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154383 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154386 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154388 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:14.157498 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154391 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154394 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154398 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154400 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154403 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154406 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154409 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154411 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154414 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154416 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154419 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154422 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154424 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154426 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154429 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154432 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154435 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154438 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154441 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154443 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:14.157979 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154446 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154449 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154452 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154454 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154457 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154459 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154462 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154466 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154468 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154471 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154474 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154478 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154482 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154485 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154488 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154491 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154493 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154496 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154499 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154501 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:14.158475 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154504 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154506 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154508 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154511 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154513 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154515 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154518 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154520 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154523 2577 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154528 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154531 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154534 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154536 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154539 2577 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154542 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154544 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154547 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154549 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154552 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:14.158985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154555 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154557 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154560 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154562 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154565 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154568 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154570 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154573 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154575 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154578 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154580 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154583 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154585 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154588 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154590 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154593 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154595 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154598 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154600 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154602 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:14.159454 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154605 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154607 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154611 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154614 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154617 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154619 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154622 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154626 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154628 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154631 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154634 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154636 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154639 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154642 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154645 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154647 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154650 2577 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154652 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.154655 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154755 2577 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 04:45:14.159933 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154765 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154773 2577 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154781 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154786 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154789 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154793 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154797 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154801 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154804 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154807 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154811 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154814 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154817 2577 flags.go:64] FLAG: --cgroup-root="" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154820 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154824 2577 flags.go:64] FLAG: --client-ca-file="" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154827 2577 flags.go:64] FLAG: --cloud-provider="external" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154831 2577 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154836 2577 flags.go:64] FLAG: --cluster-domain="" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154839 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154845 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154848 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154852 2577 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154856 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 04:45:14.160444 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154860 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154863 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154866 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154869 2577 flags.go:64] FLAG: --contention-profiling="false" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154873 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154876 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154879 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154882 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154886 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154889 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154892 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154895 2577 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154898 2577 flags.go:64] FLAG: --enable-server="true" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154901 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154906 2577 flags.go:64] FLAG: --event-burst="100" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154909 2577 flags.go:64] FLAG: --event-qps="50" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154912 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154915 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154918 2577 flags.go:64] FLAG: --eviction-hard="" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154923 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154926 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154930 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154933 2577 flags.go:64] FLAG: --eviction-soft="" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154938 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154941 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 04:45:14.161091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154944 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154947 2577 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154950 2577 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154954 2577 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154958 2577 flags.go:64] FLAG: --feature-gates="" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154962 2577 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154966 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154969 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154972 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154975 2577 flags.go:64] FLAG: --healthz-port="10248" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154978 2577 flags.go:64] FLAG: --help="false" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154982 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154985 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154988 2577 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154991 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154995 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.154998 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155001 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155004 2577 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155007 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155010 2577 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155013 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155016 2577 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155019 2577 flags.go:64] FLAG: --kube-reserved="" Nov 04 04:45:14.161727 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155022 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155025 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155028 2577 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155031 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155033 2577 flags.go:64] FLAG: --lock-file="" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155036 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155041 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155044 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155049 2577 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155053 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155056 2577 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155060 2577 flags.go:64] FLAG: --logging-format="text" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155064 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155068 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155071 2577 flags.go:64] FLAG: --manifest-url="" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155074 2577 flags.go:64] FLAG: --manifest-url-header="" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155078 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155081 2577 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155085 2577 flags.go:64] FLAG: --max-pods="110" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155088 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155092 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155095 2577 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155098 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155101 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155104 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 04:45:14.162352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155107 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155114 2577 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155117 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155137 2577 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155141 2577 flags.go:64] FLAG: --pod-cidr="" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155144 2577 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155147 2577 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155150 2577 flags.go:64] FLAG: --pods-per-core="0" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155158 2577 flags.go:64] FLAG: --port="10250" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155161 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155164 2577 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-091b9b57cd9069b27" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155168 2577 flags.go:64] FLAG: --qos-reserved="" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155171 2577 flags.go:64] FLAG: --read-only-port="10255" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155176 2577 flags.go:64] FLAG: --register-node="true" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155179 2577 flags.go:64] FLAG: --register-with-taints="" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155183 2577 flags.go:64] FLAG: --registry-burst="10" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155186 2577 flags.go:64] FLAG: --registry-qps="5" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155189 2577 flags.go:64] FLAG: --reserved-cpus="" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155193 2577 flags.go:64] FLAG: --reserved-memory="" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155198 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155201 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155205 2577 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155208 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155211 2577 flags.go:64] FLAG: --runonce="false" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155214 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155217 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155220 2577 flags.go:64] FLAG: --seccomp-default="false" Nov 04 04:45:14.162944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155223 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155226 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155229 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155232 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155235 2577 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155238 2577 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155241 2577 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155244 2577 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155247 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155250 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155253 2577 flags.go:64] FLAG: --system-cgroups="" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155256 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155262 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155265 2577 flags.go:64] FLAG: --tls-cert-file="" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155268 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155273 2577 flags.go:64] FLAG: --tls-min-version="" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155276 2577 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155279 2577 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155283 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155286 2577 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155289 2577 flags.go:64] FLAG: --v="2" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155294 2577 flags.go:64] FLAG: --version="false" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155298 2577 flags.go:64] FLAG: --vmodule="" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155303 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155307 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 04:45:14.163588 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155444 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155447 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155451 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155453 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155456 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155459 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155461 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155464 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155466 2577 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155469 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155471 2577 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155474 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155476 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155479 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155481 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155484 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155487 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155489 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155492 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155494 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:14.164229 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155497 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155500 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155502 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155505 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155507 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155512 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155514 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155517 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155520 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155522 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155526 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155529 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155532 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155535 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155538 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155540 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155543 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155545 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155548 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155550 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:14.164705 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155553 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155555 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155558 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155561 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155563 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155565 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155570 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155573 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155577 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155580 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155584 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155587 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155590 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155593 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155596 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155599 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155601 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155605 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155608 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:14.165269 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155610 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155613 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155615 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155619 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155622 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155624 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155627 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155631 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155633 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155636 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155638 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155641 2577 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155643 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155646 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155649 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155651 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155654 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155656 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155659 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:14.165721 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155661 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155665 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155667 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155670 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155672 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155675 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155678 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155680 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155683 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155685 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155688 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155690 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155693 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155695 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155698 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155700 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155704 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155707 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155710 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:14.166175 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155712 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155715 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155718 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155720 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155723 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155725 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155728 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155730 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155733 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.155760 2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155797 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155801 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155805 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155809 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155811 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155814 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:14.166633 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155817 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155820 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155823 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155826 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155828 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155831 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155835 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155839 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155842 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155846 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155850 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155853 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155855 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155858 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155861 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155864 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155867 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155870 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155873 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:14.167148 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155875 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155878 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155881 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155883 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155886 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155889 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155891 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155894 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155897 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155899 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155902 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155904 2577 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155907 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155909 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155912 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155915 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155917 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155920 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155922 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:14.167883 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155925 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155927 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155930 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155932 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155935 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155937 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155940 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155942 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155946 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155949 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155951 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155954 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155956 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155960 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155962 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155965 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155967 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155970 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155973 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:14.168413 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155975 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155978 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155980 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155983 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155985 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155988 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155990 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155994 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.155999 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156001 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156004 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156007 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156009 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156012 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156014 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156017 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156019 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156022 2577 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156025 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156027 2577 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:14.169064 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156030 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156033 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156036 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156039 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156041 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156044 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156046 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156049 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156052 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156055 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156057 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156061 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156063 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156065 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156068 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156071 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156073 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156076 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156078 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:14.169571 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156082 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156085 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156088 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.156092 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.163357 2577 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.163371 2577 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.164151 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.164158 2577 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164214 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164220 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164224 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164227 2577 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164230 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164233 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164236 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164240 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164243 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164245 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164248 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:14.170078 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164251 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164254 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164257 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164260 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164262 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164265 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164268 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164270 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164273 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164276 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164279 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164283 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164286 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164289 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164292 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164295 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164297 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164301 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164304 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:14.170632 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164306 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164309 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164311 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164314 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164317 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164319 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164322 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164324 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164327 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164330 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164332 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164335 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164337 2577 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164340 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164343 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164346 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164348 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164351 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164353 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164356 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:14.171093 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164359 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164362 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164364 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164367 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164369 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164372 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164375 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164377 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164380 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164382 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164385 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164388 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164391 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164393 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164396 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164398 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164401 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164403 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164406 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164408 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:14.171590 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164411 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164413 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164416 2577 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164418 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164421 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164423 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164426 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164428 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164431 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164434 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164436 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164439 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164442 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164445 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164447 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164450 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164453 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164455 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164458 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:14.172066 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164461 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164465 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164468 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164470 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164473 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164475 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164478 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164480 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164483 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164485 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164487 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164490 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164493 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164495 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164498 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164500 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:14.172524 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164502 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.164531 2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164655 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164659 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164662 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164665 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164668 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164671 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164673 2577 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164676 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164678 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164681 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164683 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164686 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164688 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164691 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:14.172904 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164694 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164697 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164699 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164702 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164704 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164707 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164711 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164715 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164718 2577 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164729 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164732 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164735 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164738 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164740 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164743 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164745 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164748 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164751 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164753 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:14.173333 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164756 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164758 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164761 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164763 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164766 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164769 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164772 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164774 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164777 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164780 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164782 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164785 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164787 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164790 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164793 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164796 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164798 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164801 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164803 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164806 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:14.173785 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164808 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164811 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164813 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164816 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164818 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164821 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164823 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164826 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164828 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164831 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164834 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164836 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164839 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164842 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164844 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164847 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164849 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164852 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164855 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164857 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:14.174280 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164860 2577 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164862 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164865 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164867 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164870 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164873 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164875 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164877 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164881 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164884 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164888 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164891 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164894 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164897 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164900 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164903 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164905 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164908 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164911 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:14.174773 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164913 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164916 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164918 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164921 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164923 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164926 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164928 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164931 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164933 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164935 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164938 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164941 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164944 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164946 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.164968 2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.164998 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:14.175281 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165002 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165004 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165008 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165012 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165015 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165018 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165020 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165023 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165025 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165028 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165030 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165033 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165035 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165038 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165040 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165043 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165045 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165048 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165052 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:14.175674 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165055 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165058 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165060 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165063 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165065 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165068 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165070 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165073 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165075 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165078 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165081 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165083 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165086 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165088 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165091 2577 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165094 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165096 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165098 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165101 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165103 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:14.176139 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165106 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165109 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165111 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165114 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165117 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165119 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165137 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165140 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165144 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165148 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165151 2577 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165156 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165158 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165161 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165164 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165166 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165169 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165171 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165175 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:14.176604 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165177 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165180 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165182 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165186 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165189 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165192 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165194 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165197 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165199 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165202 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165204 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165207 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165209 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165212 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165214 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165217 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165219 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165222 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165225 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165227 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:14.177112 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165230 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165233 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165235 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165238 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165242 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165245 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165247 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165250 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165253 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165256 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165258 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165261 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165263 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165266 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165269 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165271 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165279 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165282 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165285 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165287 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:14.177808 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165290 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165292 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165295 2577 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165297 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165300 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165303 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:14.165306 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.165356 2577 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.169893 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.171075 2577 server.go:1014] "Starting client certificate rotation" Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.171148 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.171194 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.173323 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 04:45:14.178291 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.174427 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 04:45:14.184714 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.184698 2577 log.go:25] "Validated CRI v1 runtime API" Nov 04 04:45:14.188633 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.188619 2577 log.go:25] "Validated CRI v1 image API" Nov 04 04:45:14.189486 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.189466 2577 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 04:45:14.192907 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.192885 2577 fs.go:135] Filesystem UUIDs: map[5df3bbc1-7915-41d3-af3c-26d165331813:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d017aece-eb0b-47fc-8f0d-b0ec0e6736de:/dev/nvme0n1p4] Nov 04 04:45:14.192981 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.192903 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Nov 04 04:45:14.198885 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.198770 2577 manager.go:217] Machine: {Timestamp:2026-11-04 04:45:14.197255958 +0000 UTC m=+0.379501145 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099351 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a2a5c37c7e29e790d8c742a9b33d2 SystemUUID:ec2a2a5c-37c7-e29e-790d-8c742a9b33d2 BootID:a7603a86-74f2-49af-b6ed-cf666b70ba29 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:c9:ea:95:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:c9:ea:95:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:61:e0:fd:6c:9e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 04:45:14.198885 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.198883 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 04:45:14.198995 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.198952 2577 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 04:45:14.200894 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.200874 2577 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 04:45:14.201039 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.200895 2577 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-242.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 04:45:14.201083 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.201041 2577 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 04:45:14.201083 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.201049 2577 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 04:45:14.201083 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.201074 2577 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 04:45:14.201179 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.201091 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 04:45:14.202361 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.202350 2577 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 04:45:14.206804 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.206792 2577 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 04:45:14.206915 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.206906 2577 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 04:45:14.207745 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.207736 2577 kubelet.go:493] "Attempting to sync node with API server" Nov 04 04:45:14.207785 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.207752 2577 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 04:45:14.207785 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.207763 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 04:45:14.207785 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.207771 2577 kubelet.go:405] "Adding apiserver pod source" Nov 04 04:45:14.207785 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.207779 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 04:45:14.208228 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.208216 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 04:45:14.212238 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.212212 2577 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 04:45:14.219750 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.219738 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 04:45:14.220571 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.220559 2577 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 04:45:14.220612 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.220590 2577 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 04:45:14.220777 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.220765 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 04:45:14.220806 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.220785 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 04:45:14.220806 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.220796 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 04:45:14.220806 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.220803 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 04:45:14.220883 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.220809 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 04:45:14.220883 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.220815 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 04:45:14.221539 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.221521 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 04:45:14.221539 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.221540 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 04:45:14.221539 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.221548 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 04:45:14.221627 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.221554 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 04:45:14.221627 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.221562 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 04:45:14.221627 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.221570 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 04:45:14.223174 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.223165 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 04:45:14.223207 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.223177 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 04:45:14.227107 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.227092 2577 server.go:1263] "Started kubelet" Nov 04 04:45:14.227223 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.227194 2577 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 04:45:14.227282 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.227203 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 04:45:14.227282 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.227267 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 04:45:14.227743 ip-10-0-140-242 systemd[1]: Started Kubernetes Kubelet. Nov 04 04:45:14.228693 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.228676 2577 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 04:45:14.229967 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.229954 2577 server.go:317] "Adding debug handlers to kubelet server" Nov 04 04:45:14.233439 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.233401 2577 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Nov 04 04:45:14.235815 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.235797 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 04:45:14.236473 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.236452 2577 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 04:45:14.236731 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.236642 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:14.237378 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.237359 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 04:45:14.237378 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.237357 2577 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 04:45:14.237378 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.237364 2577 factory.go:55] Registering systemd factory Nov 04 04:45:14.237518 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.237390 2577 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 04:45:14.237518 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.237400 2577 factory.go:223] Registration of the systemd container factory successfully Nov 04 04:45:14.237518 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.237506 2577 reconstruct.go:97] "Volume reconstruction finished" Nov 04 04:45:14.237518 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.237513 2577 reconciler.go:29] "Reconciler: start to sync state" Nov 04 04:45:14.237639 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.237630 2577 factory.go:153] Registering CRI-O factory Nov 04 04:45:14.237676 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.237640 2577 factory.go:223] Registration of the crio container factory successfully Nov 04 04:45:14.237732 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.237709 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 04:45:14.237784 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.237745 2577 factory.go:103] Registering Raw factory Nov 04 04:45:14.237784 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.237760 2577 manager.go:1199] Started watching for new ooms in manager Nov 04 04:45:14.238961 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.238945 2577 manager.go:319] Starting recovery of all containers Nov 04 04:45:14.245873 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.245840 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 04:45:14.254328 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.250547 2577 manager.go:324] Recovery completed Nov 04 04:45:14.255689 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.255672 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:14.257316 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.257282 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:14.257403 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.257340 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:14.257403 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.257356 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:14.258699 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.257935 2577 cpu_manager.go:226] "Starting" policy="none" Nov 04 04:45:14.258699 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.257939 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:14.258699 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.257943 2577 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 04:45:14.258699 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.257961 2577 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 04:45:14.258905 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.258700 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:14.258905 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.258729 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:14.258905 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.258758 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:14.260222 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.260209 2577 policy_none.go:50] "Start" Nov 04 04:45:14.260284 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.260228 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 04:45:14.260284 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.260238 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 04:45:14.262261 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.262249 2577 policy_none.go:44] "Start" Nov 04 04:45:14.287699 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.287668 2577 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 04:45:14.290414 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.290399 2577 manager.go:341] "Starting Device Plugin manager" Nov 04 04:45:14.290496 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.290426 2577 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 04:45:14.290496 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.290439 2577 server.go:87] "Starting device plugin registration server" Nov 04 04:45:14.290636 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.290624 2577 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 04:45:14.290690 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.290635 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 04:45:14.290738 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.290714 2577 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 04:45:14.290822 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.290808 2577 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 04:45:14.290822 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.290819 2577 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 04:45:14.291308 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.291290 2577 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 04:45:14.291397 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.291338 2577 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:14.365615 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.365482 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 04:45:14.367356 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.365714 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-242.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 04 04:45:14.367356 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.365729 2577 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-242.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Nov 04 04:45:14.367356 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.365706 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-242.us-west-2.compute.internal.18e4be0ce723b1c8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-242.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-242.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 04:45:14.227061192 +0000 UTC m=+0.409306379,LastTimestamp:2026-11-04 04:45:14.227061192 +0000 UTC m=+0.409306379,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-242.us-west-2.compute.internal,}" Nov 04 04:45:14.367356 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.366477 2577 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 04:45:14.367356 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.366507 2577 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 04:45:14.367356 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.366546 2577 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Nov 04 04:45:14.390918 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.390903 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:14.391683 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.391668 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:14.391748 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.391702 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:14.391748 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.391713 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:14.391748 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.391734 2577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.419580 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.419531 2577 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Nov 04 04:45:14.419580 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.419495 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-242.us-west-2.compute.internal.18e4be0ce8f1746c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-242.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-242.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-242.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 04:45:14.257323116 +0000 UTC m=+0.439568309,LastTimestamp:2026-11-04 04:45:14.257323116 +0000 UTC m=+0.439568309,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-242.us-west-2.compute.internal,}" Nov 04 04:45:14.419706 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.419538 2577 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Nov 04 04:45:14.419706 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.419523 2577 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-242.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Nov 04 04:45:14.467148 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.467109 2577 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal"] Nov 04 04:45:14.467205 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.467185 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:14.467931 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.467913 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:14.467991 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.467942 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:14.467991 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.467952 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:14.470031 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.470019 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:14.470185 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.470167 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.470226 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.470214 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:14.470674 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.470660 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:14.470746 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.470681 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:14.470746 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.470692 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:14.470746 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.470697 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:14.470746 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.470722 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:14.470746 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.470738 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:14.472654 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.472638 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.472746 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.472675 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:14.473430 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.473413 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:14.473505 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.473444 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:14.473505 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.473457 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:14.490039 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.490018 2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 04:45:14.490097 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.490066 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hd8m6" Nov 04 04:45:14.509438 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.509420 2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" node="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.512131 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.512105 2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" node="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.514421 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.514399 2577 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" Nov 04 04:45:14.514480 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.514376 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-242.us-west-2.compute.internal.18e4be0ce8f1d3c8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-242.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-242.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-242.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 04:45:14.257347528 +0000 UTC m=+0.439592719,LastTimestamp:2026-11-04 04:45:14.257347528 +0000 UTC m=+0.439592719,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-242.us-west-2.compute.internal,}" Nov 04 04:45:14.538212 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.538193 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85dfaf93b35e741480ab992c223ae102-config\") pod \"kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"85dfaf93b35e741480ab992c223ae102\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.538278 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.538214 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c7105a6f9939c228417dda3a14245b0d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"c7105a6f9939c228417dda3a14245b0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.538278 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.538230 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7105a6f9939c228417dda3a14245b0d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"c7105a6f9939c228417dda3a14245b0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.579846 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.579825 2577 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-140-242.us-west-2.compute.internal\" not found" node="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.639158 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.639116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85dfaf93b35e741480ab992c223ae102-config\") pod \"kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"85dfaf93b35e741480ab992c223ae102\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.639158 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.639163 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c7105a6f9939c228417dda3a14245b0d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"c7105a6f9939c228417dda3a14245b0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.639278 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.639178 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7105a6f9939c228417dda3a14245b0d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"c7105a6f9939c228417dda3a14245b0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.639278 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.639227 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7105a6f9939c228417dda3a14245b0d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"c7105a6f9939c228417dda3a14245b0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.639278 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.639234 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c7105a6f9939c228417dda3a14245b0d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"c7105a6f9939c228417dda3a14245b0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.639278 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.639261 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85dfaf93b35e741480ab992c223ae102-config\") pod \"kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"85dfaf93b35e741480ab992c223ae102\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.669924 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.669875 2577 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-242.us-west-2.compute.internal\" not found" node="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.779957 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.779933 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:14.780970 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.780954 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:14.781032 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.780986 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:14.781032 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.781002 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:14.781032 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.781032 2577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.813051 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.813035 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.815184 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.815171 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.837902 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:14.837886 2577 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:14.837984 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:14.837906 2577 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-242.us-west-2.compute.internal\": node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:16.307101 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:16.307043 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:16.407800 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:16.407776 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:16.508587 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:16.508558 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:16.522717 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:16.522696 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Nov 04 04:45:16.522950 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:16.522856 2577 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://afa22c3b1a41d46a3a14deb84ca41375-9501ff5f8f77d258.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-140-242.us-west-2.compute.internal.18e4be0ce8f1746c\": read tcp 10.0.140.242:51224->3.20.72.244:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-140-242.us-west-2.compute.internal.18e4be0ce8f1746c default 5091 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-242.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-242.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-242.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 04:45:14 +0000 UTC,LastTimestamp:2026-11-04 04:45:14.467929371 +0000 UTC m=+0.650174558,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-242.us-west-2.compute.internal,}" Nov 04 04:45:16.609613 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:16.609561 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:16.710156 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:16.710134 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:16.810296 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:16.810272 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:16.846259 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:16.846220 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 04:40:15 +0000 UTC" deadline="2028-04-05 14:40:49.403908405 +0000 UTC" Nov 04 04:45:16.846259 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:16.846254 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12441h55m32.557657963s" Nov 04 04:45:16.878343 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:16.878293 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:16.879287 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:16.879272 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:16.879389 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:16.879302 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:16.879389 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:16.879312 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:16.910383 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:16.910361 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:17.010502 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:17.010478 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:17.010590 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.010545 2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 04:45:17.037892 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.037859 2577 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 04:45:17.088654 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.088634 2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:17.155719 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.155667 2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 04:45:17.156480 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.156459 2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:17.221462 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.221437 2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 04:45:17.318138 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:17.318097 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7105a6f9939c228417dda3a14245b0d.slice/crio-c19af60553a809d66f26d220a6a82e879bbad6051749947e5d56c0c1d5f62673 WatchSource:0}: Error finding container c19af60553a809d66f26d220a6a82e879bbad6051749947e5d56c0c1d5f62673: Status 404 returned error can't find the container with id c19af60553a809d66f26d220a6a82e879bbad6051749947e5d56c0c1d5f62673 Nov 04 04:45:17.318484 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:17.318462 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85dfaf93b35e741480ab992c223ae102.slice/crio-9be3f1a7033f5f8ce3554042a3faa48702891092bf3a36b598e0e4daf81e4d0a WatchSource:0}: Error finding container 9be3f1a7033f5f8ce3554042a3faa48702891092bf3a36b598e0e4daf81e4d0a: Status 404 returned error can't find the container with id 9be3f1a7033f5f8ce3554042a3faa48702891092bf3a36b598e0e4daf81e4d0a Nov 04 04:45:17.322347 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.322333 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 04:45:17.407150 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.407105 2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 04:45:17.493692 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.493662 2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 04:45:17.561101 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.561075 2577 apiserver.go:52] "Watching apiserver" Nov 04 04:45:17.589263 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.589240 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Nov 04 04:45:17.694416 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.694352 2577 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 04:45:17.694680 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.694661 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-stlg9","openshift-cluster-node-tuning-operator/tuned-jkgz4","openshift-multus/multus-additional-cni-plugins-mvpj9","openshift-network-operator/iptables-alerter-7n7vn","openshift-ovn-kubernetes/ovnkube-node-j7pdj","kube-system/konnectivity-agent-8zd4x","kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v","openshift-dns/node-resolver-b4n8j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal","openshift-multus/network-metrics-daemon-m87zm","openshift-network-diagnostics/network-check-target-z99g5","openshift-multus/multus-lfwvp","openshift-image-registry/node-ca-n278n"] Nov 04 04:45:17.698461 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.698443 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.700455 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.700434 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.702590 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.702569 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.704712 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.704667 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:17.707028 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.707010 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.709428 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.709157 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:17.712028 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.712000 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:17.712436 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:17.712401 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:17.714735 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.714715 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.716957 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.716939 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:17.719294 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.719274 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:17.721523 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.721506 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:17.721619 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:17.721571 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:17.723843 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.723801 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" event={"ID":"c7105a6f9939c228417dda3a14245b0d","Type":"ContainerStarted","Data":"c19af60553a809d66f26d220a6a82e879bbad6051749947e5d56c0c1d5f62673"} Nov 04 04:45:17.724064 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.723889 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:17.724151 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:17.724113 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:17.726253 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.726234 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" event={"ID":"85dfaf93b35e741480ab992c223ae102","Type":"ContainerStarted","Data":"9be3f1a7033f5f8ce3554042a3faa48702891092bf3a36b598e0e4daf81e4d0a"} Nov 04 04:45:17.757198 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.757178 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 04:45:17.757198 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.757189 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 04:45:17.757385 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.757318 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 04:45:17.757385 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.757338 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-gwbb5\"" Nov 04 04:45:17.757385 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.757361 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 04:45:17.758852 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.758837 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 04:45:17.758926 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.758888 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 04:45:17.758926 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.758893 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-l2xwd\"" Nov 04 04:45:17.760847 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.760830 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 04:45:17.760944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.760933 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ss4dp\"" Nov 04 04:45:17.761319 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.761307 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 04:45:17.762944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.762928 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 04:45:17.762987 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.762952 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 04:45:17.762987 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.762972 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kjpq6\"" Nov 04 04:45:17.763044 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.763008 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 04:45:17.765198 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.765186 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-77ccl\"" Nov 04 04:45:17.765242 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.765209 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 04:45:17.765392 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.765377 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 04:45:17.765430 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.765414 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 04:45:17.765621 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.765606 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 04:45:17.765658 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.765607 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 04:45:17.765658 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.765611 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 04:45:17.767665 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.767652 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-95fm7\"" Nov 04 04:45:17.767726 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.767708 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 04:45:17.767807 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.767796 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 04:45:17.790564 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.790546 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 04:45:17.802019 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.801998 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfb82\" (UniqueName: \"kubernetes.io/projected/b2224b84-c10c-46e8-b7ba-844bc5802f34-kube-api-access-bfb82\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:17.802077 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802023 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.802077 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802041 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-daemon-config\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.802077 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-run\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.802077 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802071 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2224b84-c10c-46e8-b7ba-844bc5802f34-host-slash\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:17.802195 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802122 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-cni-bin\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.802195 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802153 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-log-socket\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.802256 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802196 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.802256 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802225 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-run-netns\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.802256 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802250 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-cni-bin\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.802342 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802265 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tstz6\" (UniqueName: \"kubernetes.io/projected/4affa996-4479-46e2-bde8-dba1989da65d-kube-api-access-tstz6\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.802342 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802293 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b2224b84-c10c-46e8-b7ba-844bc5802f34-iptables-alerter-script\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:17.802342 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802314 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.802342 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802334 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-socket-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.802450 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802363 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b94ca77d-354d-4404-99a9-dd6dada01c24-hosts-file\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:17.802450 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802388 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-etc-kubernetes\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.802450 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802408 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-var-lib-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.802450 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802423 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovn-node-metrics-cert\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.802450 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802449 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovnkube-script-lib\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.802586 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802480 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-registration-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.802586 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802505 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-device-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.802586 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802522 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-serviceca\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:17.802586 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802550 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-modprobe-d\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.802586 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802577 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.802759 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802595 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-run-ovn-kubernetes\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.802759 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802610 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-host\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:17.802759 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802633 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr56b\" (UniqueName: \"kubernetes.io/projected/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-kube-api-access-hr56b\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:17.802759 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802648 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysctl-d\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.802759 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802668 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-var-lib-kubelet\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.802759 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802693 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-ovn\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.802759 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:17.802759 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802734 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed101df4-83a6-42e0-8638-8b99890a8553-konnectivity-ca\") pod \"konnectivity-agent-8zd4x\" (UID: \"ed101df4-83a6-42e0-8638-8b99890a8553\") " pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:17.802759 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802759 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-systemd\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.803022 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802778 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-sys\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.803022 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802797 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-system-cni-dir\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.803022 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802830 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-slash\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.803022 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802859 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-systemd\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.803022 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802900 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-node-log\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.803022 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802936 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-sys-fs\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.803022 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.802976 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-socket-dir-parent\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.803022 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803005 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-host\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.803260 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803030 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-etc-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.803260 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803080 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knj8p\" (UniqueName: \"kubernetes.io/projected/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-kube-api-access-knj8p\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.803260 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803099 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovnkube-config\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.803260 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803113 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b94ca77d-354d-4404-99a9-dd6dada01c24-tmp-dir\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:17.803260 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803133 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad2cc585-2e96-43fe-b778-f19bb57ac50d-cni-binary-copy\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.803260 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803150 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-cni-multus\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.803260 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803173 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-kubernetes\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.803260 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803219 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.803260 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803270 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnjmq\" (UniqueName: \"kubernetes.io/projected/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-kube-api-access-dnjmq\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803293 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803316 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ddd38197-2fcb-4086-a376-b5ef4628be3c-kubelet-config\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803337 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-k8s-cni-cncf-io\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803356 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-netns\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803375 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-conf-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803409 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-systemd-units\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803459 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ddd38197-2fcb-4086-a376-b5ef4628be3c-dbus\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803475 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgt87\" (UniqueName: \"kubernetes.io/projected/53eaa748-5fc9-4d33-8fd4-946c97ec0074-kube-api-access-dgt87\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803490 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-cnibin\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803504 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-multus-certs\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nbph\" (UniqueName: \"kubernetes.io/projected/ad2cc585-2e96-43fe-b778-f19bb57ac50d-kube-api-access-4nbph\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.803560 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803546 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-lib-modules\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803578 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4affa996-4479-46e2-bde8-dba1989da65d-etc-tuned\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803597 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-kubelet\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803614 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-env-overrides\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803634 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf6dc\" (UniqueName: \"kubernetes.io/projected/b94ca77d-354d-4404-99a9-dd6dada01c24-kube-api-access-sf6dc\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803647 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-hostroot\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803669 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed101df4-83a6-42e0-8638-8b99890a8553-agent-certs\") pod \"konnectivity-agent-8zd4x\" (UID: \"ed101df4-83a6-42e0-8638-8b99890a8553\") " pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803683 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s778k\" (UniqueName: \"kubernetes.io/projected/d282c492-0fef-42cf-922e-86b1355221f2-kube-api-access-s778k\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803696 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-cni-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803714 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4affa996-4479-46e2-bde8-dba1989da65d-tmp\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cnibin\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803744 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-cni-netd\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803779 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-etc-selinux\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803810 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-system-cni-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803839 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysconfig\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803881 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysctl-conf\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803904 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:17.803962 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803930 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-kubelet\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.804449 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.803951 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v85z\" (UniqueName: \"kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z\") pod \"network-check-target-z99g5\" (UID: \"09c6a70b-bc11-4b06-a052-ca93a697355b\") " pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:17.853182 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.853156 2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 04:45:17.853297 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.853216 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6rgx8" Nov 04 04:45:17.858141 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.858112 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 04:45:17.858194 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.858149 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 04:45:17.858271 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.858232 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 04:45:17.858324 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.858274 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 04:45:17.858493 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.858469 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 04:45:17.858493 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.858474 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 04:45:17.858604 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.858526 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 04:45:17.859075 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.859057 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7qqph\"" Nov 04 04:45:17.859075 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.859079 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 04:45:17.859227 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.859108 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8xbd4\"" Nov 04 04:45:17.859543 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.859521 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nhf59\"" Nov 04 04:45:17.904881 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.904847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ddd38197-2fcb-4086-a376-b5ef4628be3c-dbus\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:17.904989 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.904894 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgt87\" (UniqueName: \"kubernetes.io/projected/53eaa748-5fc9-4d33-8fd4-946c97ec0074-kube-api-access-dgt87\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:17.904989 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.904912 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-cnibin\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.905067 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905020 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-multus-certs\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.905067 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905040 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nbph\" (UniqueName: \"kubernetes.io/projected/ad2cc585-2e96-43fe-b778-f19bb57ac50d-kube-api-access-4nbph\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.905067 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-lib-modules\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.905186 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905081 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4affa996-4479-46e2-bde8-dba1989da65d-etc-tuned\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.905186 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905104 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-kubelet\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.905186 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905108 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-multus-certs\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.905186 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905127 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-env-overrides\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.905186 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905150 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf6dc\" (UniqueName: \"kubernetes.io/projected/b94ca77d-354d-4404-99a9-dd6dada01c24-kube-api-access-sf6dc\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:17.905186 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905172 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-hostroot\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905184 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-kubelet\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905194 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed101df4-83a6-42e0-8638-8b99890a8553-agent-certs\") pod \"konnectivity-agent-8zd4x\" (UID: \"ed101df4-83a6-42e0-8638-8b99890a8553\") " pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s778k\" (UniqueName: \"kubernetes.io/projected/d282c492-0fef-42cf-922e-86b1355221f2-kube-api-access-s778k\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905261 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-cni-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905277 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4affa996-4479-46e2-bde8-dba1989da65d-tmp\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905270 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-cnibin\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905297 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cnibin\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905337 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cnibin\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905340 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ddd38197-2fcb-4086-a376-b5ef4628be3c-dbus\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905336 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-cni-netd\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905373 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-cni-netd\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905374 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-etc-selinux\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905389 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-cni-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905407 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-system-cni-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905435 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysconfig\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905444 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-etc-selinux\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905450 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-hostroot\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.905463 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905460 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysctl-conf\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905489 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905508 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-system-cni-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905516 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysconfig\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905518 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-kubelet\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905527 2577 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905559 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v85z\" (UniqueName: \"kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z\") pod \"network-check-target-z99g5\" (UID: \"09c6a70b-bc11-4b06-a052-ca93a697355b\") " pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905584 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfb82\" (UniqueName: \"kubernetes.io/projected/b2224b84-c10c-46e8-b7ba-844bc5802f34-kube-api-access-bfb82\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905615 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905561 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-kubelet\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905638 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-daemon-config\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905664 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-run\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:17.905667 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905675 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905685 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2224b84-c10c-46e8-b7ba-844bc5802f34-host-slash\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905684 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-env-overrides\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:17.905754 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs podName:53eaa748-5fc9-4d33-8fd4-946c97ec0074 nodeName:}" failed. No retries permitted until 2026-11-04 04:45:18.405713102 +0000 UTC m=+3.236213183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs") pod "network-metrics-daemon-m87zm" (UID: "53eaa748-5fc9-4d33-8fd4-946c97ec0074") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 04:45:17.906218 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905733 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-run\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905638 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysctl-conf\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905800 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2224b84-c10c-46e8-b7ba-844bc5802f34-host-slash\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905796 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-cni-bin\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905854 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-log-socket\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905864 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-cni-bin\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905907 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905920 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-log-socket\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905927 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-run-netns\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905957 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-run-netns\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905958 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-cni-bin\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905978 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-lib-modules\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.905985 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tstz6\" (UniqueName: \"kubernetes.io/projected/4affa996-4479-46e2-bde8-dba1989da65d-kube-api-access-tstz6\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b2224b84-c10c-46e8-b7ba-844bc5802f34-iptables-alerter-script\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906022 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-cni-bin\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906035 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906059 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-socket-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.907009 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906083 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b94ca77d-354d-4404-99a9-dd6dada01c24-hosts-file\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-etc-kubernetes\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906139 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906149 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-var-lib-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906172 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovn-node-metrics-cert\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906196 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovnkube-script-lib\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906239 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-var-lib-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906252 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-daemon-config\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906294 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b94ca77d-354d-4404-99a9-dd6dada01c24-hosts-file\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906327 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-etc-kubernetes\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906353 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-registration-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906380 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-device-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906384 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-socket-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906402 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-serviceca\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906424 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-modprobe-d\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906447 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906469 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-registration-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.907778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906503 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-run-ovn-kubernetes\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906564 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-modprobe-d\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906566 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-host\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906598 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr56b\" (UniqueName: \"kubernetes.io/projected/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-kube-api-access-hr56b\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906608 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-device-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906635 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysctl-d\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906657 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-var-lib-kubelet\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906679 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-ovn\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906702 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906726 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed101df4-83a6-42e0-8638-8b99890a8553-konnectivity-ca\") pod \"konnectivity-agent-8zd4x\" (UID: \"ed101df4-83a6-42e0-8638-8b99890a8553\") " pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906728 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b2224b84-c10c-46e8-b7ba-844bc5802f34-iptables-alerter-script\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906748 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-systemd\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906772 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-sys\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906800 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-system-cni-dir\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906808 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-var-lib-kubelet\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906823 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-slash\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.908606 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906846 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-systemd\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906853 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-host\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906889 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-systemd\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906899 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-node-log\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906915 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-systemd\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-sys-fs\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906936 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-ovn\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906960 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-socket-dir-parent\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906970 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-sys\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907001 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-host\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907008 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-socket-dir-parent\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907027 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-system-cni-dir\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907035 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-serviceca\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.906822 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovnkube-script-lib\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907054 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-host\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907055 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907093 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-slash\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907115 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-node-log\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.909435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907146 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysctl-d\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907150 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-sys-fs\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:17.907161 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907175 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-etc-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907201 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knj8p\" (UniqueName: \"kubernetes.io/projected/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-kube-api-access-knj8p\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:17.907224 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret podName:ddd38197-2fcb-4086-a376-b5ef4628be3c nodeName:}" failed. No retries permitted until 2026-11-04 04:45:18.407208838 +0000 UTC m=+3.237708933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret") pod "global-pull-secret-syncer-stlg9" (UID: "ddd38197-2fcb-4086-a376-b5ef4628be3c") : object "kube-system"/"original-pull-secret" not registered Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907216 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-run-ovn-kubernetes\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907260 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovnkube-config\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907288 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b94ca77d-354d-4404-99a9-dd6dada01c24-tmp-dir\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907314 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad2cc585-2e96-43fe-b778-f19bb57ac50d-cni-binary-copy\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907340 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-cni-multus\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907347 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-etc-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907362 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-kubernetes\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907396 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907449 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnjmq\" (UniqueName: \"kubernetes.io/projected/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-kube-api-access-dnjmq\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907476 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.910116 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907500 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ddd38197-2fcb-4086-a376-b5ef4628be3c-kubelet-config\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907523 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-k8s-cni-cncf-io\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907548 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-netns\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907549 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed101df4-83a6-42e0-8638-8b99890a8553-konnectivity-ca\") pod \"konnectivity-agent-8zd4x\" (UID: \"ed101df4-83a6-42e0-8638-8b99890a8553\") " pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907571 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-conf-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907601 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-systemd-units\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907678 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-systemd-units\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907683 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b94ca77d-354d-4404-99a9-dd6dada01c24-tmp-dir\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907693 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907753 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ddd38197-2fcb-4086-a376-b5ef4628be3c-kubelet-config\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907767 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-netns\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907801 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-conf-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907805 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-k8s-cni-cncf-io\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907842 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovnkube-config\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907850 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-cni-multus\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907936 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-kubernetes\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.907948 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.908312 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad2cc585-2e96-43fe-b778-f19bb57ac50d-cni-binary-copy\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.910584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.908314 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.911097 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.909273 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4affa996-4479-46e2-bde8-dba1989da65d-tmp\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.911097 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.909307 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4affa996-4479-46e2-bde8-dba1989da65d-etc-tuned\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.911097 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.909406 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovn-node-metrics-cert\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:17.911097 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.910093 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed101df4-83a6-42e0-8638-8b99890a8553-agent-certs\") pod \"konnectivity-agent-8zd4x\" (UID: \"ed101df4-83a6-42e0-8638-8b99890a8553\") " pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:17.969969 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:17.969891 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 04:45:17.969969 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:17.969914 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 04:45:17.969969 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:17.969925 2577 projected.go:196] Error preparing data for projected volume kube-api-access-9v85z for pod openshift-network-diagnostics/network-check-target-z99g5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 04:45:17.970212 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:17.969982 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z podName:09c6a70b-bc11-4b06-a052-ca93a697355b nodeName:}" failed. No retries permitted until 2026-11-04 04:45:18.469964531 +0000 UTC m=+3.300464616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9v85z" (UniqueName: "kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z") pod "network-check-target-z99g5" (UID: "09c6a70b-bc11-4b06-a052-ca93a697355b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 04:45:17.973091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.972690 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nbph\" (UniqueName: \"kubernetes.io/projected/ad2cc585-2e96-43fe-b778-f19bb57ac50d-kube-api-access-4nbph\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:17.973091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.972754 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s778k\" (UniqueName: \"kubernetes.io/projected/d282c492-0fef-42cf-922e-86b1355221f2-kube-api-access-s778k\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:17.973091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.972763 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgt87\" (UniqueName: \"kubernetes.io/projected/53eaa748-5fc9-4d33-8fd4-946c97ec0074-kube-api-access-dgt87\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:17.973091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.972791 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfb82\" (UniqueName: \"kubernetes.io/projected/b2224b84-c10c-46e8-b7ba-844bc5802f34-kube-api-access-bfb82\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:17.973091 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.973041 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf6dc\" (UniqueName: \"kubernetes.io/projected/b94ca77d-354d-4404-99a9-dd6dada01c24-kube-api-access-sf6dc\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:17.973403 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.973349 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tstz6\" (UniqueName: \"kubernetes.io/projected/4affa996-4479-46e2-bde8-dba1989da65d-kube-api-access-tstz6\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:17.974806 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.974468 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr56b\" (UniqueName: \"kubernetes.io/projected/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-kube-api-access-hr56b\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:17.974806 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.974759 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnjmq\" (UniqueName: \"kubernetes.io/projected/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-kube-api-access-dnjmq\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:17.974806 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:17.974774 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knj8p\" (UniqueName: \"kubernetes.io/projected/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-kube-api-access-knj8p\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:18.010905 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.010885 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lfwvp" Nov 04 04:45:18.016311 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.016297 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:18.020565 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:18.020544 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad2cc585_2e96_43fe_b778_f19bb57ac50d.slice/crio-6978674a56f0167af3ec8a283d5a8fe6aefab5a734b4a8300bbac7b835634a4f WatchSource:0}: Error finding container 6978674a56f0167af3ec8a283d5a8fe6aefab5a734b4a8300bbac7b835634a4f: Status 404 returned error can't find the container with id 6978674a56f0167af3ec8a283d5a8fe6aefab5a734b4a8300bbac7b835634a4f Nov 04 04:45:18.024972 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:18.024952 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4affa996_4479_46e2_bde8_dba1989da65d.slice/crio-63e7282e98c02ead71a065e1b3746b201e60f95d4fd5e3be6235bca2e0b602ba WatchSource:0}: Error finding container 63e7282e98c02ead71a065e1b3746b201e60f95d4fd5e3be6235bca2e0b602ba: Status 404 returned error can't find the container with id 63e7282e98c02ead71a065e1b3746b201e60f95d4fd5e3be6235bca2e0b602ba Nov 04 04:45:18.036035 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.036019 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:18.040796 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.040778 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:18.042957 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.042934 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:18.044663 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:18.044645 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2224b84_c10c_46e8_b7ba_844bc5802f34.slice/crio-f7fbd87646d42f04b25d855a9e927485ad1afcf375b28062cd94a0165f1024fd WatchSource:0}: Error finding container f7fbd87646d42f04b25d855a9e927485ad1afcf375b28062cd94a0165f1024fd: Status 404 returned error can't find the container with id f7fbd87646d42f04b25d855a9e927485ad1afcf375b28062cd94a0165f1024fd Nov 04 04:45:18.051223 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:18.051203 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod456c901b_f6fa_4de0_85aa_59d4fc1ec5c0.slice/crio-b152b518b2c4ab500c6c8a655d1ef9e058ad29779fd75aa42f9f654c005e0e28 WatchSource:0}: Error finding container b152b518b2c4ab500c6c8a655d1ef9e058ad29779fd75aa42f9f654c005e0e28: Status 404 returned error can't find the container with id b152b518b2c4ab500c6c8a655d1ef9e058ad29779fd75aa42f9f654c005e0e28 Nov 04 04:45:18.051766 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:18.051745 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded101df4_83a6_42e0_8638_8b99890a8553.slice/crio-383877d45ee0186dfdd340a869fad974454cc6cf3815715a9630fe719adca3e9 WatchSource:0}: Error finding container 383877d45ee0186dfdd340a869fad974454cc6cf3815715a9630fe719adca3e9: Status 404 returned error can't find the container with id 383877d45ee0186dfdd340a869fad974454cc6cf3815715a9630fe719adca3e9 Nov 04 04:45:18.057058 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.057044 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:18.059208 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.059188 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:18.061714 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.061701 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:18.065781 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.065761 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:18.065985 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:18.065967 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94ca77d_354d_4404_99a9_dd6dada01c24.slice/crio-610459a2adf085c6e38b56ac3824da2ffa4455257dc5d30899cd56a26931c9a7 WatchSource:0}: Error finding container 610459a2adf085c6e38b56ac3824da2ffa4455257dc5d30899cd56a26931c9a7: Status 404 returned error can't find the container with id 610459a2adf085c6e38b56ac3824da2ffa4455257dc5d30899cd56a26931c9a7 Nov 04 04:45:18.070206 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:18.070184 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd282c492_0fef_42cf_922e_86b1355221f2.slice/crio-66d254dc707f52bcd5471b2162264ed4d0cc8ca74201d666c3b3ec51ee45ed15 WatchSource:0}: Error finding container 66d254dc707f52bcd5471b2162264ed4d0cc8ca74201d666c3b3ec51ee45ed15: Status 404 returned error can't find the container with id 66d254dc707f52bcd5471b2162264ed4d0cc8ca74201d666c3b3ec51ee45ed15 Nov 04 04:45:18.070658 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:18.070619 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58c2628d_e8f4_4dd4_987e_c7c4f3847fac.slice/crio-d6f34ef8de03254b9160071bb40a3e2495fd3b63b8c61b3a702f88b38efddb03 WatchSource:0}: Error finding container d6f34ef8de03254b9160071bb40a3e2495fd3b63b8c61b3a702f88b38efddb03: Status 404 returned error can't find the container with id d6f34ef8de03254b9160071bb40a3e2495fd3b63b8c61b3a702f88b38efddb03 Nov 04 04:45:18.075209 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:18.075177 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd6b4c99_03f6_42fb_9a97_ff51e2efb90c.slice/crio-9d2bbdbccf9a47a129c2066af32da04db445970994c1e69c54df165fb91c4a91 WatchSource:0}: Error finding container 9d2bbdbccf9a47a129c2066af32da04db445970994c1e69c54df165fb91c4a91: Status 404 returned error can't find the container with id 9d2bbdbccf9a47a129c2066af32da04db445970994c1e69c54df165fb91c4a91 Nov 04 04:45:18.411906 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.411821 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:18.412055 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.411911 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:18.412055 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:18.411980 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 04:45:18.412055 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:18.412032 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 04:45:18.412055 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:18.412040 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs podName:53eaa748-5fc9-4d33-8fd4-946c97ec0074 nodeName:}" failed. No retries permitted until 2026-11-04 04:45:19.412023712 +0000 UTC m=+4.242523796 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs") pod "network-metrics-daemon-m87zm" (UID: "53eaa748-5fc9-4d33-8fd4-946c97ec0074") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 04:45:18.412299 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:18.412082 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret podName:ddd38197-2fcb-4086-a376-b5ef4628be3c nodeName:}" failed. No retries permitted until 2026-11-04 04:45:19.412066996 +0000 UTC m=+4.242567066 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret") pod "global-pull-secret-syncer-stlg9" (UID: "ddd38197-2fcb-4086-a376-b5ef4628be3c") : object "kube-system"/"original-pull-secret" not registered Nov 04 04:45:18.513177 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.513148 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v85z\" (UniqueName: \"kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z\") pod \"network-check-target-z99g5\" (UID: \"09c6a70b-bc11-4b06-a052-ca93a697355b\") " pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:18.513351 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:18.513330 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 04:45:18.513405 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:18.513358 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 04:45:18.513405 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:18.513371 2577 projected.go:196] Error preparing data for projected volume kube-api-access-9v85z for pod openshift-network-diagnostics/network-check-target-z99g5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 04:45:18.513479 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:18.513426 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z podName:09c6a70b-bc11-4b06-a052-ca93a697355b nodeName:}" failed. No retries permitted until 2026-11-04 04:45:19.513412655 +0000 UTC m=+4.343912717 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9v85z" (UniqueName: "kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z") pod "network-check-target-z99g5" (UID: "09c6a70b-bc11-4b06-a052-ca93a697355b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 04:45:18.727507 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.727408 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" event={"ID":"4affa996-4479-46e2-bde8-dba1989da65d","Type":"ContainerStarted","Data":"63e7282e98c02ead71a065e1b3746b201e60f95d4fd5e3be6235bca2e0b602ba"} Nov 04 04:45:18.728680 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.728652 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lfwvp" event={"ID":"ad2cc585-2e96-43fe-b778-f19bb57ac50d","Type":"ContainerStarted","Data":"6978674a56f0167af3ec8a283d5a8fe6aefab5a734b4a8300bbac7b835634a4f"} Nov 04 04:45:18.729910 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.729886 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" event={"ID":"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c","Type":"ContainerStarted","Data":"9d2bbdbccf9a47a129c2066af32da04db445970994c1e69c54df165fb91c4a91"} Nov 04 04:45:18.731076 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.731030 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n278n" event={"ID":"58c2628d-e8f4-4dd4-987e-c7c4f3847fac","Type":"ContainerStarted","Data":"d6f34ef8de03254b9160071bb40a3e2495fd3b63b8c61b3a702f88b38efddb03"} Nov 04 04:45:18.732459 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.732431 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" event={"ID":"d282c492-0fef-42cf-922e-86b1355221f2","Type":"ContainerStarted","Data":"66d254dc707f52bcd5471b2162264ed4d0cc8ca74201d666c3b3ec51ee45ed15"} Nov 04 04:45:18.734102 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.734078 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b4n8j" event={"ID":"b94ca77d-354d-4404-99a9-dd6dada01c24","Type":"ContainerStarted","Data":"610459a2adf085c6e38b56ac3824da2ffa4455257dc5d30899cd56a26931c9a7"} Nov 04 04:45:18.735564 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.735539 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8zd4x" event={"ID":"ed101df4-83a6-42e0-8638-8b99890a8553","Type":"ContainerStarted","Data":"383877d45ee0186dfdd340a869fad974454cc6cf3815715a9630fe719adca3e9"} Nov 04 04:45:18.736803 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.736778 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvpj9" event={"ID":"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0","Type":"ContainerStarted","Data":"b152b518b2c4ab500c6c8a655d1ef9e058ad29779fd75aa42f9f654c005e0e28"} Nov 04 04:45:18.738149 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.738128 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7n7vn" event={"ID":"b2224b84-c10c-46e8-b7ba-844bc5802f34","Type":"ContainerStarted","Data":"f7fbd87646d42f04b25d855a9e927485ad1afcf375b28062cd94a0165f1024fd"} Nov 04 04:45:18.856466 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.856412 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 04:40:17 +0000 UTC" deadline="2028-04-18 12:08:29.821637361 +0000 UTC" Nov 04 04:45:18.856466 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:18.856462 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12751h23m10.965178161s" Nov 04 04:45:19.426251 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:19.426211 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:19.426486 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:19.426316 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:19.426486 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:19.426436 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 04:45:19.426486 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:19.426495 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret podName:ddd38197-2fcb-4086-a376-b5ef4628be3c nodeName:}" failed. No retries permitted until 2026-11-04 04:45:21.426476114 +0000 UTC m=+6.256976192 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret") pod "global-pull-secret-syncer-stlg9" (UID: "ddd38197-2fcb-4086-a376-b5ef4628be3c") : object "kube-system"/"original-pull-secret" not registered Nov 04 04:45:19.426694 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:19.426623 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 04:45:19.426694 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:19.426657 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs podName:53eaa748-5fc9-4d33-8fd4-946c97ec0074 nodeName:}" failed. No retries permitted until 2026-11-04 04:45:21.426645305 +0000 UTC m=+6.257145369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs") pod "network-metrics-daemon-m87zm" (UID: "53eaa748-5fc9-4d33-8fd4-946c97ec0074") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 04:45:19.527274 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:19.527235 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v85z\" (UniqueName: \"kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z\") pod \"network-check-target-z99g5\" (UID: \"09c6a70b-bc11-4b06-a052-ca93a697355b\") " pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:19.527459 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:19.527393 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 04:45:19.527459 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:19.527417 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 04:45:19.527459 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:19.527429 2577 projected.go:196] Error preparing data for projected volume kube-api-access-9v85z for pod openshift-network-diagnostics/network-check-target-z99g5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 04:45:19.527622 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:19.527486 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z podName:09c6a70b-bc11-4b06-a052-ca93a697355b nodeName:}" failed. No retries permitted until 2026-11-04 04:45:21.52746914 +0000 UTC m=+6.357969225 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9v85z" (UniqueName: "kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z") pod "network-check-target-z99g5" (UID: "09c6a70b-bc11-4b06-a052-ca93a697355b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 04:45:19.719777 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:19.719707 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:19.719944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:19.719780 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:19.719944 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:19.719799 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:19.720095 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:19.720039 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:19.720169 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:19.720115 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:19.720221 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:19.720171 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:19.857526 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:19.857484 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 04:40:17 +0000 UTC" deadline="2028-06-29 22:18:49.127627653 +0000 UTC" Nov 04 04:45:19.857526 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:19.857525 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14489h33m29.270106967s" Nov 04 04:45:21.445630 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:21.445386 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:21.446196 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:21.445696 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:21.446196 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:21.445837 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 04:45:21.446196 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:21.445921 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret podName:ddd38197-2fcb-4086-a376-b5ef4628be3c nodeName:}" failed. No retries permitted until 2026-11-04 04:45:25.445901975 +0000 UTC m=+10.276402059 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret") pod "global-pull-secret-syncer-stlg9" (UID: "ddd38197-2fcb-4086-a376-b5ef4628be3c") : object "kube-system"/"original-pull-secret" not registered Nov 04 04:45:21.446196 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:21.446001 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 04:45:21.446196 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:21.446039 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs podName:53eaa748-5fc9-4d33-8fd4-946c97ec0074 nodeName:}" failed. No retries permitted until 2026-11-04 04:45:25.44602686 +0000 UTC m=+10.276526924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs") pod "network-metrics-daemon-m87zm" (UID: "53eaa748-5fc9-4d33-8fd4-946c97ec0074") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 04:45:21.546105 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:21.546026 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v85z\" (UniqueName: \"kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z\") pod \"network-check-target-z99g5\" (UID: \"09c6a70b-bc11-4b06-a052-ca93a697355b\") " pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:21.546263 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:21.546176 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 04:45:21.546263 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:21.546196 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 04:45:21.546263 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:21.546209 2577 projected.go:196] Error preparing data for projected volume kube-api-access-9v85z for pod openshift-network-diagnostics/network-check-target-z99g5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 04:45:21.546414 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:21.546263 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z podName:09c6a70b-bc11-4b06-a052-ca93a697355b nodeName:}" failed. No retries permitted until 2026-11-04 04:45:25.546244584 +0000 UTC m=+10.376744651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9v85z" (UniqueName: "kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z") pod "network-check-target-z99g5" (UID: "09c6a70b-bc11-4b06-a052-ca93a697355b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 04:45:21.723406 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:21.723380 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:21.723553 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:21.723525 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:21.723625 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:21.723602 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:21.723890 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:21.723679 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:21.724193 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:21.724154 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:21.724550 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:21.724499 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:21.769832 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:21.769772 2577 generic.go:356] "Generic (PLEG): container finished" podID="c7105a6f9939c228417dda3a14245b0d" containerID="c3c1ce9b24df4e674e18b39b34ff88edbbe188e6e5016a586ce2c066f54e9363" exitCode=0 Nov 04 04:45:21.769995 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:21.769881 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" event={"ID":"c7105a6f9939c228417dda3a14245b0d","Type":"ContainerDied","Data":"c3c1ce9b24df4e674e18b39b34ff88edbbe188e6e5016a586ce2c066f54e9363"} Nov 04 04:45:21.780480 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:21.780384 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" event={"ID":"85dfaf93b35e741480ab992c223ae102","Type":"ContainerStarted","Data":"cb403ef0eeb644a1e803e1b994eb249f9b51cd4b87bc9fe437f8517de4810220"} Nov 04 04:45:22.801156 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:22.800235 2577 generic.go:356] "Generic (PLEG): container finished" podID="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" containerID="cb112106c49102d35c0fd31620ed7934c1a535f2e75294a6e0c96484da340322" exitCode=0 Nov 04 04:45:22.801156 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:22.800362 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvpj9" event={"ID":"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0","Type":"ContainerDied","Data":"cb112106c49102d35c0fd31620ed7934c1a535f2e75294a6e0c96484da340322"} Nov 04 04:45:22.831050 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:22.830973 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" event={"ID":"c7105a6f9939c228417dda3a14245b0d","Type":"ContainerStarted","Data":"96ddf95e63cc4c6abb70d34a2d161a91fe8902f7cb298d93716d8f1afa12b5bf"} Nov 04 04:45:23.099799 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:23.099682 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" podStartSLOduration=6.099643552 podStartE2EDuration="6.099643552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:22.024015108 +0000 UTC m=+6.854515192" watchObservedRunningTime="2026-11-04 04:45:23.099643552 +0000 UTC m=+7.930143640" Nov 04 04:45:23.721008 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:23.720973 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:23.721182 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:23.721095 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:23.721182 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:23.721160 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:23.721296 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:23.721221 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:23.721296 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:23.721262 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:23.721398 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:23.721313 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:23.842077 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:23.842023 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8zd4x" event={"ID":"ed101df4-83a6-42e0-8638-8b99890a8553","Type":"ContainerStarted","Data":"3c8f1fef5212f35a050eaf6f0defca983294d088f81a16f5b93850ca3001da1a"} Nov 04 04:45:23.969357 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:23.969302 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" podStartSLOduration=6.969283163 podStartE2EDuration="6.969283163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:23.172544655 +0000 UTC m=+8.003044738" watchObservedRunningTime="2026-11-04 04:45:23.969283163 +0000 UTC m=+8.799783248" Nov 04 04:45:24.847676 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:24.847546 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n278n" event={"ID":"58c2628d-e8f4-4dd4-987e-c7c4f3847fac","Type":"ContainerStarted","Data":"9e313185d5822aaca412d05e1124eaa5466a18d687f00b84427af26e3ae8db3e"} Nov 04 04:45:24.851452 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:24.851365 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" event={"ID":"d282c492-0fef-42cf-922e-86b1355221f2","Type":"ContainerStarted","Data":"0f5e79787e908f7cd903a5c2343971fcb056c93b43bb01007463bb009678c345"} Nov 04 04:45:24.975313 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:24.973927 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8zd4x" podStartSLOduration=4.287389316 podStartE2EDuration="8.973909058s" totalImagesPullingTime="4.686519742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:23.969246949 +0000 UTC m=+8.799747033" watchObservedRunningTime="2026-11-04 04:45:24.973909058 +0000 UTC m=+9.804409141" Nov 04 04:45:25.480263 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:25.480229 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:25.480424 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:25.480306 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:25.480495 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:25.480457 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 04:45:25.480554 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:25.480516 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs podName:53eaa748-5fc9-4d33-8fd4-946c97ec0074 nodeName:}" failed. No retries permitted until 2026-11-04 04:45:33.480498632 +0000 UTC m=+18.310998707 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs") pod "network-metrics-daemon-m87zm" (UID: "53eaa748-5fc9-4d33-8fd4-946c97ec0074") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 04:45:25.480617 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:25.480579 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 04:45:25.480617 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:25.480610 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret podName:ddd38197-2fcb-4086-a376-b5ef4628be3c nodeName:}" failed. No retries permitted until 2026-11-04 04:45:33.480599722 +0000 UTC m=+18.311099797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret") pod "global-pull-secret-syncer-stlg9" (UID: "ddd38197-2fcb-4086-a376-b5ef4628be3c") : object "kube-system"/"original-pull-secret" not registered Nov 04 04:45:25.580973 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:25.580674 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v85z\" (UniqueName: \"kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z\") pod \"network-check-target-z99g5\" (UID: \"09c6a70b-bc11-4b06-a052-ca93a697355b\") " pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:25.580973 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:25.580824 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 04:45:25.580973 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:25.580846 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 04:45:25.580973 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:25.580860 2577 projected.go:196] Error preparing data for projected volume kube-api-access-9v85z for pod openshift-network-diagnostics/network-check-target-z99g5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 04:45:25.580973 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:25.580927 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z podName:09c6a70b-bc11-4b06-a052-ca93a697355b nodeName:}" failed. No retries permitted until 2026-11-04 04:45:33.580910447 +0000 UTC m=+18.411410526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9v85z" (UniqueName: "kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z") pod "network-check-target-z99g5" (UID: "09c6a70b-bc11-4b06-a052-ca93a697355b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 04:45:25.721225 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:25.720717 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:25.721225 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:25.720830 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:25.721225 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:25.720906 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:25.721225 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:25.720961 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:25.721225 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:25.721004 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:25.721225 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:25.721055 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:26.351927 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:26.351859 2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:26.353079 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:26.353053 2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:26.481489 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:26.481108 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-n278n" podStartSLOduration=4.763730756 podStartE2EDuration="10.481087678s" totalImagesPullingTime="5.717356922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:24.975240886 +0000 UTC m=+9.805740970" watchObservedRunningTime="2026-11-04 04:45:26.481087678 +0000 UTC m=+11.311587762" Nov 04 04:45:26.855401 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:26.855353 2577 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 04:45:26.860084 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:26.858436 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lfwvp" event={"ID":"ad2cc585-2e96-43fe-b778-f19bb57ac50d","Type":"ContainerStarted","Data":"89c55e18b4527a6ab9619a41c31826b56b5d791624d7b3d42615b80a9d1ef7ca"} Nov 04 04:45:26.861559 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:26.861530 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" event={"ID":"d282c492-0fef-42cf-922e-86b1355221f2","Type":"ContainerStarted","Data":"453782ecb55ebceddf9120dd40954ffedf621ecab2c66e2a4a8f7be88a4ccfa8"} Nov 04 04:45:26.863404 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:26.863376 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b4n8j" event={"ID":"b94ca77d-354d-4404-99a9-dd6dada01c24","Type":"ContainerStarted","Data":"8136927d925d9f813451f3b23a5e0e45264551f59571e09af9352f75a5cf9914"} Nov 04 04:45:26.988761 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:26.988672 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-lfwvp" podStartSLOduration=2.552974132 podStartE2EDuration="10.988653757s" totalImagesPullingTime="8.435679625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:26.988217876 +0000 UTC m=+11.818717961" watchObservedRunningTime="2026-11-04 04:45:26.988653757 +0000 UTC m=+11.819153834" Nov 04 04:45:27.115836 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:27.115789 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-b4n8j" podStartSLOduration=2.649267657 podStartE2EDuration="11.115773613s" totalImagesPullingTime="8.466505956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:27.115297709 +0000 UTC m=+11.945797795" watchObservedRunningTime="2026-11-04 04:45:27.115773613 +0000 UTC m=+11.946273696" Nov 04 04:45:27.648045 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:27.647767 2577 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T04:45:26.855374253Z","UUID":"10e317f6-9b7b-4b43-b0e0-23d4d9abb572","Handler":null,"Name":"","Endpoint":""} Nov 04 04:45:27.649757 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:27.649382 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 04:45:27.649757 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:27.649412 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 04:45:27.719024 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:27.718997 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:27.719170 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:27.719102 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:27.719170 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:27.719162 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:27.719292 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:27.719227 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:27.719377 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:27.719359 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:27.719466 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:27.719448 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:27.867522 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:27.867485 2577 generic.go:356] "Generic (PLEG): container finished" podID="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" containerID="dcf2f25d282df7d33e799b70792fcb4cbbd6fb842f355f819fcebc1720e0b688" exitCode=0 Nov 04 04:45:27.867678 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:27.867601 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvpj9" event={"ID":"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0","Type":"ContainerDied","Data":"dcf2f25d282df7d33e799b70792fcb4cbbd6fb842f355f819fcebc1720e0b688"} Nov 04 04:45:28.872288 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:28.872200 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" event={"ID":"d282c492-0fef-42cf-922e-86b1355221f2","Type":"ContainerStarted","Data":"6996321ba11d807169a789f7ecb333573667063f82d6c42d02b7072f63f79cf8"} Nov 04 04:45:28.874054 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:28.874023 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7n7vn" event={"ID":"b2224b84-c10c-46e8-b7ba-844bc5802f34","Type":"ContainerStarted","Data":"0b8f231147a86ba911cb1ff6c2363b8ef8612c97e2c0b562d2f9cde90542439b"} Nov 04 04:45:29.003590 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:29.003539 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" podStartSLOduration=3.092244155 podStartE2EDuration="13.003526883s" totalImagesPullingTime="9.911282728s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:45:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:29.002919928 +0000 UTC m=+13.833420011" watchObservedRunningTime="2026-11-04 04:45:29.003526883 +0000 UTC m=+13.834026966" Nov 04 04:45:29.133798 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:29.133674 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7n7vn" podStartSLOduration=4.284462201 podStartE2EDuration="13.133654744s" totalImagesPullingTime="8.849192543s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:29.133387305 +0000 UTC m=+13.963887391" watchObservedRunningTime="2026-11-04 04:45:29.133654744 +0000 UTC m=+13.964154830" Nov 04 04:45:29.718895 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:29.718775 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:29.719048 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:29.718936 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:29.719384 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:29.719358 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:29.719482 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:29.719455 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:29.719636 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:29.719611 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:29.719727 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:29.719703 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:29.878441 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:29.878404 2577 generic.go:356] "Generic (PLEG): container finished" podID="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" containerID="0a371c35877d97b12124a2510b733eb4940f0c1a8dffcf2d9ba76d11f997eee7" exitCode=0 Nov 04 04:45:29.879027 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:29.878498 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvpj9" event={"ID":"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0","Type":"ContainerDied","Data":"0a371c35877d97b12124a2510b733eb4940f0c1a8dffcf2d9ba76d11f997eee7"} Nov 04 04:45:29.880505 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:29.880473 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" event={"ID":"4affa996-4479-46e2-bde8-dba1989da65d","Type":"ContainerStarted","Data":"f96cf50a5bdba8d65d7b1c6f789deecbdbb720f8d571db2c36cda9cd7d77de18"} Nov 04 04:45:30.141657 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:30.141568 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" podStartSLOduration=3.145037709 podStartE2EDuration="14.141554781s" totalImagesPullingTime="10.996517072s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:30.141004257 +0000 UTC m=+14.971504342" watchObservedRunningTime="2026-11-04 04:45:30.141554781 +0000 UTC m=+14.972054864" Nov 04 04:45:31.719569 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:31.719544 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:31.720155 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:31.719596 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:31.720155 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:31.719609 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:31.720155 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:31.719678 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:31.720155 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:31.719744 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:31.720155 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:31.719851 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:31.887467 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:31.887227 2577 generic.go:356] "Generic (PLEG): container finished" podID="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" containerID="66162b4fee7f656de1905d6a7715e2f361b08ae219cb9a86de66c9d3dc8c8431" exitCode=0 Nov 04 04:45:31.887467 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:31.887316 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvpj9" event={"ID":"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0","Type":"ContainerDied","Data":"66162b4fee7f656de1905d6a7715e2f361b08ae219cb9a86de66c9d3dc8c8431"} Nov 04 04:45:33.539920 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:33.539891 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:33.540338 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:33.539965 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:33.540338 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:33.540069 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 04:45:33.540338 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:33.540097 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 04:45:33.540338 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:33.540143 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs podName:53eaa748-5fc9-4d33-8fd4-946c97ec0074 nodeName:}" failed. No retries permitted until 2026-11-04 04:45:49.540121465 +0000 UTC m=+34.370621547 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs") pod "network-metrics-daemon-m87zm" (UID: "53eaa748-5fc9-4d33-8fd4-946c97ec0074") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 04:45:33.540338 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:33.540164 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret podName:ddd38197-2fcb-4086-a376-b5ef4628be3c nodeName:}" failed. No retries permitted until 2026-11-04 04:45:49.540153617 +0000 UTC m=+34.370653694 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret") pod "global-pull-secret-syncer-stlg9" (UID: "ddd38197-2fcb-4086-a376-b5ef4628be3c") : object "kube-system"/"original-pull-secret" not registered Nov 04 04:45:33.640663 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:33.640640 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v85z\" (UniqueName: \"kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z\") pod \"network-check-target-z99g5\" (UID: \"09c6a70b-bc11-4b06-a052-ca93a697355b\") " pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:33.640826 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:33.640807 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 04:45:33.640899 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:33.640837 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 04:45:33.640899 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:33.640852 2577 projected.go:196] Error preparing data for projected volume kube-api-access-9v85z for pod openshift-network-diagnostics/network-check-target-z99g5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 04:45:33.640979 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:33.640924 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z podName:09c6a70b-bc11-4b06-a052-ca93a697355b nodeName:}" failed. No retries permitted until 2026-11-04 04:45:49.64090723 +0000 UTC m=+34.471407305 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9v85z" (UniqueName: "kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z") pod "network-check-target-z99g5" (UID: "09c6a70b-bc11-4b06-a052-ca93a697355b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 04:45:33.719704 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:33.719354 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:33.719704 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:33.719380 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:33.719704 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:33.719669 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:33.719704 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:33.719441 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:33.719976 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:33.719744 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:33.719976 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:33.719827 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:33.895771 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:33.895685 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" event={"ID":"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c","Type":"ContainerStarted","Data":"6f70a0761d5e6cdfcb36a8bf5c9aa6aa9abc7a92c2d79ce0a8f2950eb0767ccc"} Nov 04 04:45:33.895771 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:33.895767 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" event={"ID":"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c","Type":"ContainerStarted","Data":"41e2dac4aed71a57278c8b8ac96f47765107a74e71f077d8acc84044bc63f762"} Nov 04 04:45:33.895973 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:33.895803 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" event={"ID":"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c","Type":"ContainerStarted","Data":"435ee271a2b56076718d595b5b89e3ed20fa5e8be3c5eeed584a90d79dbaab8c"} Nov 04 04:45:33.895973 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:33.895846 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" event={"ID":"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c","Type":"ContainerStarted","Data":"f11c2eea77b03cc740c7efb3e80340152422744b0b9e4eebdf098b7aa33b9b4f"} Nov 04 04:45:33.895973 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:33.895898 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" event={"ID":"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c","Type":"ContainerStarted","Data":"4b29d2c9d4d0d2649af7f9a107a906b574059dbec0d528bd29e80472dd4a90e7"} Nov 04 04:45:33.895973 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:33.895928 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" event={"ID":"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c","Type":"ContainerStarted","Data":"b0d29f9c22d806ad2df70da7df411334fc5c42ba72a917a11cf2a13b6492d5af"} Nov 04 04:45:34.900795 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:34.900762 2577 generic.go:356] "Generic (PLEG): container finished" podID="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" containerID="6ee384c6c84bc2eae9d72dfecab4914909f3541d2f5ecca54773c4a35a6f0a54" exitCode=0 Nov 04 04:45:34.901220 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:34.900816 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvpj9" event={"ID":"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0","Type":"ContainerDied","Data":"6ee384c6c84bc2eae9d72dfecab4914909f3541d2f5ecca54773c4a35a6f0a54"} Nov 04 04:45:35.719300 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:35.719276 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:35.719420 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:35.719377 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:35.719420 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:35.719398 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:35.719498 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:35.719483 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:35.719541 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:35.719391 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:35.719625 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:35.719608 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:35.905381 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:35.905348 2577 generic.go:356] "Generic (PLEG): container finished" podID="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" containerID="4a0d9396befb3530a130c0c8cb28b1595d4a40e17308863d9c865e3b2a42e1b4" exitCode=0 Nov 04 04:45:35.905958 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:35.905430 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvpj9" event={"ID":"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0","Type":"ContainerDied","Data":"4a0d9396befb3530a130c0c8cb28b1595d4a40e17308863d9c865e3b2a42e1b4"} Nov 04 04:45:35.908836 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:35.908815 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" event={"ID":"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c","Type":"ContainerStarted","Data":"6492fcf9558fa341c38cd2826b50d72a5072bfe950a4579050a0b932d65ae4a7"} Nov 04 04:45:36.351419 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:36.351388 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:36.351584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:36.351504 2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 04:45:36.351952 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:36.351932 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:36.914584 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:36.914552 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvpj9" event={"ID":"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0","Type":"ContainerStarted","Data":"e9c9b51abdcaf251a6bbc882658cf21b6f98d672be491073c7121ad5a2ce6bc5"} Nov 04 04:45:37.049907 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:37.049842 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mvpj9" podStartSLOduration=7.420085638 podStartE2EDuration="21.049830138s" totalImagesPullingTime="13.6297445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:45:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:37.04958094 +0000 UTC m=+21.880081025" watchObservedRunningTime="2026-11-04 04:45:37.049830138 +0000 UTC m=+21.880330221" Nov 04 04:45:37.719252 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:37.719214 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:37.719252 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:37.719239 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:37.719435 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:37.719404 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:37.719435 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:37.719433 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:37.719618 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:37.719527 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:37.719675 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:37.719645 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:37.919821 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:37.919788 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" event={"ID":"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c","Type":"ContainerStarted","Data":"50ee6501591d2f3109cb1e6d00753058fb28ae22e161c95bd5da9ad09f436dfa"} Nov 04 04:45:38.157423 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:38.157241 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" podStartSLOduration=6.912852358 podStartE2EDuration="22.157226526s" totalImagesPullingTime="15.244374168s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:38.119830442 +0000 UTC m=+22.950330528" watchObservedRunningTime="2026-11-04 04:45:38.157226526 +0000 UTC m=+22.987726609" Nov 04 04:45:38.205186 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:38.205156 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:38.399699 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:38.399668 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:38.921980 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:38.921943 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:38.922420 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:38.922022 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:38.922420 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:38.922064 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:38.937270 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:38.937249 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:38.937383 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:38.937373 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:39.719519 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:39.719485 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:39.719519 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:39.719521 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:39.719791 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:39.719485 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:39.719791 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:39.719602 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:39.719791 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:39.719660 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:39.720157 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:39.720123 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:40.400535 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:40.400462 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z99g5"] Nov 04 04:45:40.401005 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:40.400581 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:40.401005 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:40.400696 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:40.403357 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:40.403338 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-stlg9"] Nov 04 04:45:40.403468 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:40.403431 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:40.403543 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:40.403526 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:40.403821 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:40.403803 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m87zm"] Nov 04 04:45:40.403914 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:40.403902 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:40.403993 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:40.403979 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:41.718782 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:41.718708 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:41.719141 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:41.718715 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:41.719141 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:41.718813 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:41.719141 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:41.718930 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:42.120909 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:42.120864 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:42.260418 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:42.260366 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:42.719292 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:42.719256 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:42.719630 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:42.719361 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:43.719177 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:43.719144 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:43.719177 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:43.719169 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:43.719711 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:43.719291 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:43.719711 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:43.719424 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:44.718981 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:44.718945 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:44.719163 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:44.719097 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:45.033412 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:45.033384 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:45.175412 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:45.175385 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:45.419428 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:45.419351 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:45.559125 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:45.559095 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:45.719757 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:45.719689 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:45.719917 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:45.719766 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-stlg9" podUID="ddd38197-2fcb-4086-a376-b5ef4628be3c" Nov 04 04:45:45.719917 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:45.719891 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:45.720035 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:45.720017 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m87zm" podUID="53eaa748-5fc9-4d33-8fd4-946c97ec0074" Nov 04 04:45:45.744224 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:45.744200 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:45.886649 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:45.886629 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:46.531535 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:46.531505 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:46.707054 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:46.707017 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:46.718985 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:46.718965 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:46.719072 ip-10-0-140-242 kubenswrapper[2577]: E1104 04:45:46.719056 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z99g5" podUID="09c6a70b-bc11-4b06-a052-ca93a697355b" Nov 04 04:45:46.871994 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:46.871925 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:47.076639 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.076609 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:47.278246 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.278220 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:47.439045 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.438989 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeReady" Nov 04 04:45:47.439150 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.439094 2577 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Nov 04 04:45:47.489302 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.489278 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:47.694565 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.694493 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:47.718611 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.718593 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:47.718778 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.718766 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:47.777999 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.777980 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m97m8\"" Nov 04 04:45:47.777999 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.777980 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-rrdk2\"" Nov 04 04:45:47.778145 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.777980 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 04:45:47.778145 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.777984 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 04:45:47.841822 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:47.841796 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:48.457130 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:48.457104 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b4n8j_b94ca77d-354d-4404-99a9-dd6dada01c24/dns-node-resolver/0.log" Nov 04 04:45:48.516556 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:48.516529 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:48.719422 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:48.719344 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:48.794954 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:48.794926 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 04:45:48.794954 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:48.794929 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-45k46\"" Nov 04 04:45:48.795159 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:48.794990 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 04:45:48.931976 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:48.931943 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:49.112379 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.112339 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:49.322693 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.322671 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n278n_58c2628d-e8f4-4dd4-987e-c7c4f3847fac/node-ca/0.log" Nov 04 04:45:49.397528 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.397470 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:49.559993 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.559966 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:49.560113 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.560007 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:49.563224 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.563196 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:49.563323 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.563246 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:49.660610 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.660554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v85z\" (UniqueName: \"kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z\") pod \"network-check-target-z99g5\" (UID: \"09c6a70b-bc11-4b06-a052-ca93a697355b\") " pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:49.662941 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.662913 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v85z\" (UniqueName: \"kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z\") pod \"network-check-target-z99g5\" (UID: \"09c6a70b-bc11-4b06-a052-ca93a697355b\") " pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:49.795344 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.795315 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:49.831383 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.831355 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:49.838524 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.838503 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:49.929503 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.929449 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:49.939804 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:49.939782 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:50.204995 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:50.204915 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:50.219728 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:50.219699 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53eaa748_5fc9_4d33_8fd4_946c97ec0074.slice/crio-9d236d115a08f5ef6ae8733f68f9f9adbad07d48e8470e84deac192c60e053f7 WatchSource:0}: Error finding container 9d236d115a08f5ef6ae8733f68f9f9adbad07d48e8470e84deac192c60e053f7: Status 404 returned error can't find the container with id 9d236d115a08f5ef6ae8733f68f9f9adbad07d48e8470e84deac192c60e053f7 Nov 04 04:45:50.225476 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:50.225451 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m87zm"] Nov 04 04:45:50.265594 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:50.265571 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-stlg9"] Nov 04 04:45:50.266170 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:50.266140 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd38197_2fcb_4086_a376_b5ef4628be3c.slice/crio-ea6e461f4948f77f3b3fbeb64dc683a3616234616994328510a59ee35e8d938a WatchSource:0}: Error finding container ea6e461f4948f77f3b3fbeb64dc683a3616234616994328510a59ee35e8d938a: Status 404 returned error can't find the container with id ea6e461f4948f77f3b3fbeb64dc683a3616234616994328510a59ee35e8d938a Nov 04 04:45:50.311441 ip-10-0-140-242 kubenswrapper[2577]: W1104 04:45:50.311420 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09c6a70b_bc11_4b06_a052_ca93a697355b.slice/crio-8fd2ec5b8981239b072f71c45c2424fc32be35d27319a476aebcdfdba705a19b WatchSource:0}: Error finding container 8fd2ec5b8981239b072f71c45c2424fc32be35d27319a476aebcdfdba705a19b: Status 404 returned error can't find the container with id 8fd2ec5b8981239b072f71c45c2424fc32be35d27319a476aebcdfdba705a19b Nov 04 04:45:50.312833 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:50.312804 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z99g5"] Nov 04 04:45:50.344509 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:50.344488 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:50.597938 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:50.597913 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:50.798725 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:50.798694 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:50.946352 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:50.946283 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z99g5" event={"ID":"09c6a70b-bc11-4b06-a052-ca93a697355b","Type":"ContainerStarted","Data":"8fd2ec5b8981239b072f71c45c2424fc32be35d27319a476aebcdfdba705a19b"} Nov 04 04:45:50.947386 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:50.947358 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-stlg9" event={"ID":"ddd38197-2fcb-4086-a376-b5ef4628be3c","Type":"ContainerStarted","Data":"ea6e461f4948f77f3b3fbeb64dc683a3616234616994328510a59ee35e8d938a"} Nov 04 04:45:50.948356 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:50.948329 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m87zm" event={"ID":"53eaa748-5fc9-4d33-8fd4-946c97ec0074","Type":"ContainerStarted","Data":"9d236d115a08f5ef6ae8733f68f9f9adbad07d48e8470e84deac192c60e053f7"} Nov 04 04:45:51.002547 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:51.002525 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:51.395325 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:51.395293 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:52.067479 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:52.067426 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:52.207542 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:52.207477 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:52.596428 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:52.596389 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:52.800216 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:52.800182 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:52.992738 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:52.992699 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:53.394520 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:53.394440 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:53.958068 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:53.958038 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m87zm" event={"ID":"53eaa748-5fc9-4d33-8fd4-946c97ec0074","Type":"ContainerStarted","Data":"cf2d677398dc23cda1b2b09c9f5b40db92f9b1a9607ec60c00f459a74c17079e"} Nov 04 04:45:53.996633 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:53.996603 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:54.596450 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:54.596420 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:54.794607 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:54.794575 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:54.963185 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:54.963101 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m87zm" event={"ID":"53eaa748-5fc9-4d33-8fd4-946c97ec0074","Type":"ContainerStarted","Data":"25775ead5252383c7c390e63245153f30f22e7732e52e36e56e141b93d572f9c"} Nov 04 04:45:54.964718 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:54.964694 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z99g5" event={"ID":"09c6a70b-bc11-4b06-a052-ca93a697355b","Type":"ContainerStarted","Data":"8099691acbdb2201ce99cb4cd591584436a76bd98debe5428c34b993f39c7c54"} Nov 04 04:45:54.964857 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:54.964837 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:55.003588 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:55.003559 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:55.091826 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:55.091776 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-m87zm" podStartSLOduration=35.587174017 podStartE2EDuration="39.09176298s" totalImagesPullingTime="3.504588963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:55.091203089 +0000 UTC m=+39.921703172" watchObservedRunningTime="2026-11-04 04:45:55.09176298 +0000 UTC m=+39.922263063" Nov 04 04:45:55.146390 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:55.146361 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:55.218637 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:55.218553 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z99g5" podStartSLOduration=34.920846184 podStartE2EDuration="39.218539657s" totalImagesPullingTime="4.297693473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:45:55.218072727 +0000 UTC m=+40.048572822" watchObservedRunningTime="2026-11-04 04:45:55.218539657 +0000 UTC m=+40.049039741" Nov 04 04:45:55.286806 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:55.286773 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:56.294217 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:56.294174 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:56.434815 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:56.434783 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:56.594960 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:56.594882 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:56.797316 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:56.797274 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:56.996035 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:56.995998 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:57.197577 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:57.197546 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:57.337229 ip-10-0-140-242 kubenswrapper[2577]: I1104 04:45:57.337155 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:45:58.217164 ip-10-0-140-242 systemd[1]: Stopping Kubernetes Kubelet... Nov 04 04:45:58.223369 ip-10-0-140-242 systemd[1]: kubelet.service: Deactivated successfully. Nov 04 04:45:58.223547 ip-10-0-140-242 systemd[1]: Stopped Kubernetes Kubelet. Nov 04 04:45:58.223595 ip-10-0-140-242 systemd[1]: kubelet.service: Consumed 2.248s CPU time. Nov 04 04:45:58.247460 ip-10-0-140-242 systemd[1]: Starting Kubernetes Kubelet... Nov 04 04:45:58.330704 ip-10-0-140-242 kubenswrapper[4374]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 04:45:58.330704 ip-10-0-140-242 kubenswrapper[4374]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 04:45:58.330704 ip-10-0-140-242 kubenswrapper[4374]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 04:45:58.330704 ip-10-0-140-242 kubenswrapper[4374]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 04:45:58.332778 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332699 4374 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:58.332778 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332775 4374 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:58.332778 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332781 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:58.332778 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332784 4374 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332787 4374 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332790 4374 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332794 4374 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332797 4374 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332800 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332804 4374 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332809 4374 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332812 4374 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332815 4374 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332818 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332821 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332824 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332827 4374 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332829 4374 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332832 4374 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332834 4374 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332837 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332848 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332851 4374 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:58.332951 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332854 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332857 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332859 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332862 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332876 4374 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332879 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332883 4374 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332888 4374 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332892 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332895 4374 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332897 4374 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332900 4374 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332903 4374 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332905 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332908 4374 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332911 4374 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332914 4374 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332917 4374 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332919 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:58.333452 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332922 4374 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332924 4374 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332927 4374 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332930 4374 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332932 4374 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332936 4374 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332938 4374 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332941 4374 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332943 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332946 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332948 4374 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332951 4374 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332954 4374 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332957 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332960 4374 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332962 4374 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332965 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332968 4374 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332971 4374 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332973 4374 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:58.333948 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332976 4374 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332979 4374 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332981 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332984 4374 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332986 4374 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332989 4374 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332992 4374 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332995 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.332997 4374 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333000 4374 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333003 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333005 4374 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333008 4374 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333011 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333013 4374 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333016 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333018 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333022 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333024 4374 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:58.334478 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333027 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333030 4374 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333033 4374 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333035 4374 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333038 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333040 4374 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333043 4374 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333046 4374 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333048 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333051 4374 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333054 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333057 4374 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333060 4374 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333063 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333065 4374 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333068 4374 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333071 4374 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333073 4374 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333076 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:58.334957 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333079 4374 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333081 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333084 4374 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333087 4374 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333090 4374 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333092 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333840 4374 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333849 4374 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333854 4374 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333857 4374 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333860 4374 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333863 4374 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333884 4374 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333890 4374 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333894 4374 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333898 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333902 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333906 4374 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333910 4374 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:58.335419 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333912 4374 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333915 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333918 4374 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333920 4374 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333923 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333927 4374 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333931 4374 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333935 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333938 4374 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333942 4374 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333945 4374 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333947 4374 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333950 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333953 4374 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333956 4374 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333959 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333961 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333964 4374 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333967 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:58.335918 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333969 4374 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333972 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333975 4374 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333977 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333980 4374 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333983 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333985 4374 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333988 4374 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333991 4374 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333994 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333997 4374 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.333999 4374 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334002 4374 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334004 4374 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334007 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334009 4374 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334012 4374 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334014 4374 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334017 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334020 4374 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:58.336398 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334022 4374 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334025 4374 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334030 4374 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334033 4374 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334035 4374 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334038 4374 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334041 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334044 4374 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334046 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334049 4374 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334051 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334054 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334058 4374 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334060 4374 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334063 4374 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334066 4374 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334068 4374 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334071 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334073 4374 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334076 4374 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:58.336902 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334078 4374 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334082 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334084 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334087 4374 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334090 4374 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334092 4374 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334095 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334099 4374 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334102 4374 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334104 4374 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334107 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334110 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334112 4374 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334115 4374 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334119 4374 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334122 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334124 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334127 4374 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334130 4374 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:58.337397 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334132 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334135 4374 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334137 4374 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334140 4374 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334142 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334145 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334148 4374 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334150 4374 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334153 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334155 4374 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334158 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334160 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334163 4374 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334165 4374 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334168 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334629 4374 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334635 4374 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334640 4374 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334644 4374 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:58.337884 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334648 4374 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334651 4374 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334654 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334657 4374 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334661 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334664 4374 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334667 4374 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334669 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334674 4374 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334677 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334680 4374 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334683 4374 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334685 4374 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334688 4374 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334691 4374 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334693 4374 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334696 4374 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334698 4374 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334701 4374 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:58.338365 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334704 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334706 4374 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334709 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334711 4374 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334714 4374 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334716 4374 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334719 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334721 4374 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334724 4374 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334727 4374 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334730 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334732 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334737 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334739 4374 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334742 4374 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334745 4374 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334748 4374 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334750 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334752 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334755 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:58.338830 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334758 4374 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334762 4374 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334764 4374 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334767 4374 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334770 4374 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334772 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334775 4374 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334778 4374 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334781 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334783 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334786 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334788 4374 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334791 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334794 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334796 4374 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334799 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334801 4374 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334804 4374 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334807 4374 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:58.339374 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334809 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334812 4374 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334814 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334817 4374 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334820 4374 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334824 4374 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334826 4374 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334829 4374 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334831 4374 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334834 4374 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334836 4374 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334838 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334841 4374 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334844 4374 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334848 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334851 4374 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334853 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334856 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334858 4374 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334861 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:58.339839 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334864 4374 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334883 4374 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334887 4374 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334891 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334895 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334899 4374 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334902 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334905 4374 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334908 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334910 4374 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334913 4374 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334915 4374 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334918 4374 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334921 4374 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334923 4374 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334925 4374 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334929 4374 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334933 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334936 4374 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334939 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:58.340355 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334941 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334943 4374 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334947 4374 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.334950 4374 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335041 4374 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335052 4374 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335061 4374 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335068 4374 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335075 4374 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335080 4374 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335084 4374 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335092 4374 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335095 4374 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335099 4374 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335102 4374 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335105 4374 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335108 4374 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335111 4374 flags.go:64] FLAG: --cgroup-root="" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335114 4374 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335117 4374 flags.go:64] FLAG: --client-ca-file="" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335120 4374 flags.go:64] FLAG: --cloud-provider="external" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335123 4374 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335127 4374 flags.go:64] FLAG: --cluster-domain="" Nov 04 04:45:58.340848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335130 4374 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335133 4374 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335136 4374 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335140 4374 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335143 4374 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335146 4374 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335151 4374 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335154 4374 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335157 4374 flags.go:64] FLAG: --contention-profiling="false" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335160 4374 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335162 4374 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335166 4374 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335168 4374 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335173 4374 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335176 4374 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335179 4374 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335182 4374 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335185 4374 flags.go:64] FLAG: --enable-server="true" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335188 4374 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335192 4374 flags.go:64] FLAG: --event-burst="100" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335195 4374 flags.go:64] FLAG: --event-qps="50" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335198 4374 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335202 4374 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335205 4374 flags.go:64] FLAG: --eviction-hard="" Nov 04 04:45:58.341422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335209 4374 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335212 4374 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335215 4374 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335218 4374 flags.go:64] FLAG: --eviction-soft="" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335221 4374 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335224 4374 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335227 4374 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335230 4374 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335233 4374 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335236 4374 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335239 4374 flags.go:64] FLAG: --feature-gates="" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335243 4374 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335246 4374 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335249 4374 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335253 4374 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335256 4374 flags.go:64] FLAG: --healthz-port="10248" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335259 4374 flags.go:64] FLAG: --help="false" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335262 4374 flags.go:64] FLAG: --hostname-override="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335266 4374 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335269 4374 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335272 4374 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335276 4374 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335279 4374 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 04:45:58.342023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335282 4374 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335285 4374 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335288 4374 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335291 4374 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335294 4374 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335297 4374 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335300 4374 flags.go:64] FLAG: --kube-reserved="" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335303 4374 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335306 4374 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335309 4374 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335312 4374 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335315 4374 flags.go:64] FLAG: --lock-file="" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335317 4374 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335320 4374 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335323 4374 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335328 4374 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335331 4374 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335334 4374 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335337 4374 flags.go:64] FLAG: --logging-format="text" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335340 4374 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335344 4374 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335346 4374 flags.go:64] FLAG: --manifest-url="" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335349 4374 flags.go:64] FLAG: --manifest-url-header="" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335354 4374 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335357 4374 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 04:45:58.342568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335362 4374 flags.go:64] FLAG: --max-pods="110" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335365 4374 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335368 4374 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335371 4374 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335374 4374 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335377 4374 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335380 4374 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335383 4374 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335390 4374 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335393 4374 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335396 4374 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335399 4374 flags.go:64] FLAG: --pod-cidr="" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335402 4374 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335404 4374 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335407 4374 flags.go:64] FLAG: --pods-per-core="0" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335410 4374 flags.go:64] FLAG: --port="10250" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335413 4374 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335416 4374 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-091b9b57cd9069b27" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335419 4374 flags.go:64] FLAG: --qos-reserved="" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335423 4374 flags.go:64] FLAG: --read-only-port="10255" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335426 4374 flags.go:64] FLAG: --register-node="true" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335429 4374 flags.go:64] FLAG: --register-with-taints="" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335433 4374 flags.go:64] FLAG: --registry-burst="10" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335436 4374 flags.go:64] FLAG: --registry-qps="5" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335439 4374 flags.go:64] FLAG: --reserved-cpus="" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335442 4374 flags.go:64] FLAG: --reserved-memory="" Nov 04 04:45:58.343197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335446 4374 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335449 4374 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335452 4374 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335455 4374 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335457 4374 flags.go:64] FLAG: --runonce="false" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335460 4374 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335463 4374 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335468 4374 flags.go:64] FLAG: --seccomp-default="false" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335471 4374 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335474 4374 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335478 4374 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335481 4374 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335484 4374 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335487 4374 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335489 4374 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335492 4374 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335495 4374 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335498 4374 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335501 4374 flags.go:64] FLAG: --system-cgroups="" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335504 4374 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335512 4374 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335515 4374 flags.go:64] FLAG: --tls-cert-file="" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335518 4374 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335523 4374 flags.go:64] FLAG: --tls-min-version="" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335526 4374 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 04:45:58.343826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335528 4374 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335531 4374 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335536 4374 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335539 4374 flags.go:64] FLAG: --v="2" Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335543 4374 flags.go:64] FLAG: --version="false" Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335547 4374 flags.go:64] FLAG: --vmodule="" Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335551 4374 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.335554 4374 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335692 4374 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335695 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335698 4374 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335702 4374 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335705 4374 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335708 4374 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335712 4374 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335715 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335718 4374 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335720 4374 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335723 4374 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335725 4374 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335728 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:58.344564 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335730 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335733 4374 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335735 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335738 4374 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335740 4374 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335743 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335746 4374 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335748 4374 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335751 4374 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335753 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335756 4374 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335759 4374 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335761 4374 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335766 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335768 4374 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335771 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335773 4374 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335776 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335778 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335782 4374 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:58.345121 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335784 4374 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335787 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335789 4374 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335792 4374 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335795 4374 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335799 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335802 4374 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335805 4374 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335807 4374 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335810 4374 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335813 4374 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335815 4374 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335818 4374 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335820 4374 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335823 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335825 4374 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335828 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335830 4374 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335833 4374 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335836 4374 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:58.346079 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335838 4374 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335841 4374 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335844 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335846 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335851 4374 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335856 4374 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335859 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335862 4374 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335876 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335879 4374 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335882 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335885 4374 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335888 4374 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335891 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335893 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335896 4374 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335899 4374 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335903 4374 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335905 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:58.346833 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335908 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335911 4374 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335914 4374 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335916 4374 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335919 4374 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335922 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335925 4374 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335927 4374 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335930 4374 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335933 4374 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335935 4374 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335938 4374 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335941 4374 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335944 4374 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335948 4374 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335951 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335954 4374 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335957 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335961 4374 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:58.347456 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335964 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335966 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335969 4374 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335971 4374 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335974 4374 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335977 4374 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335979 4374 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335982 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335985 4374 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335987 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335990 4374 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335994 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335997 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.335999 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336002 4374 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.336029 4374 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 04:45:58.348066 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336070 4374 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336074 4374 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336077 4374 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336079 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336082 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336085 4374 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336087 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336090 4374 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336093 4374 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336095 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336098 4374 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336100 4374 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336103 4374 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336106 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336108 4374 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336112 4374 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336115 4374 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336117 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336120 4374 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336123 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336125 4374 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:58.348503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336128 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336130 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336133 4374 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336136 4374 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336138 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336141 4374 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336143 4374 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336147 4374 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336150 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336153 4374 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336155 4374 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336158 4374 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336161 4374 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336164 4374 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336167 4374 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336169 4374 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336172 4374 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336174 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336177 4374 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336179 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:58.349212 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336182 4374 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336184 4374 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336187 4374 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336190 4374 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336192 4374 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336195 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336199 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336203 4374 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336207 4374 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336210 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336213 4374 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336215 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336218 4374 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336220 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336223 4374 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336226 4374 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336228 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336231 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336233 4374 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:58.349859 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336237 4374 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336240 4374 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336242 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336246 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336248 4374 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336251 4374 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336254 4374 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336257 4374 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336259 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336262 4374 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336264 4374 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336267 4374 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336270 4374 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336272 4374 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336275 4374 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336277 4374 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336280 4374 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336284 4374 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336288 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:58.350349 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336293 4374 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336296 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336298 4374 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336301 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336303 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336306 4374 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336309 4374 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336311 4374 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336314 4374 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336317 4374 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336319 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336322 4374 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336324 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336328 4374 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336331 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336333 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336336 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336338 4374 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336349 4374 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:58.350816 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336353 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336356 4374 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336358 4374 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336361 4374 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336364 4374 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336366 4374 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336369 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.336372 4374 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.339957 4374 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.339972 4374 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.339989 4374 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.339994 4374 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340058 4374 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340064 4374 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340068 4374 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340071 4374 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340074 4374 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340076 4374 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340080 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:58.351343 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340083 4374 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340086 4374 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340089 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340092 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340094 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340097 4374 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340101 4374 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340107 4374 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340113 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340119 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340124 4374 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340127 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340129 4374 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340133 4374 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340137 4374 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340141 4374 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340144 4374 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340147 4374 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:58.351952 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340150 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340153 4374 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340156 4374 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340159 4374 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340161 4374 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340164 4374 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340167 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340170 4374 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340172 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340175 4374 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340178 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340180 4374 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340183 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340186 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340189 4374 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340191 4374 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340194 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340197 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340199 4374 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340202 4374 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:58.352553 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340205 4374 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340207 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340209 4374 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340212 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340215 4374 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340217 4374 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340220 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340222 4374 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340225 4374 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340228 4374 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340230 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340233 4374 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340235 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340239 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340242 4374 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340244 4374 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340247 4374 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340249 4374 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340252 4374 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340255 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:58.353140 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340257 4374 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340260 4374 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340262 4374 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340265 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340267 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340270 4374 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340273 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340275 4374 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340278 4374 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340280 4374 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340283 4374 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340286 4374 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340288 4374 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340291 4374 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340294 4374 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340296 4374 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340299 4374 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340302 4374 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340304 4374 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:58.353634 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340307 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340309 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340313 4374 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340315 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340318 4374 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340320 4374 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340324 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340327 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340329 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340332 4374 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340334 4374 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340337 4374 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340340 4374 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340349 4374 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340352 4374 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340355 4374 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340358 4374 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340361 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340363 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340366 4374 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:58.354117 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340369 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340372 4374 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.340400 4374 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340547 4374 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340551 4374 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340554 4374 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340557 4374 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340560 4374 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340562 4374 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340565 4374 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340568 4374 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340570 4374 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340573 4374 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340575 4374 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340579 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340582 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:58.354598 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340585 4374 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340588 4374 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340591 4374 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340594 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340596 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340599 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340602 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340605 4374 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340607 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340610 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340612 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340615 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340617 4374 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340620 4374 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340623 4374 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340625 4374 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340628 4374 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340631 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340634 4374 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:58.355007 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340636 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340639 4374 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340641 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340644 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340646 4374 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340649 4374 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340652 4374 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340654 4374 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340657 4374 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340659 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340662 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340665 4374 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340668 4374 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340671 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340674 4374 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340676 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340679 4374 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340682 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340684 4374 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340687 4374 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:58.355503 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340689 4374 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340692 4374 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340694 4374 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340697 4374 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340699 4374 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340702 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340704 4374 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340707 4374 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340709 4374 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340712 4374 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340714 4374 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340717 4374 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340720 4374 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340722 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340726 4374 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340729 4374 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340732 4374 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340735 4374 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340737 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340740 4374 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:58.356002 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340743 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340745 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340748 4374 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340750 4374 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340755 4374 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340759 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340762 4374 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340765 4374 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340768 4374 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340771 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340773 4374 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340776 4374 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340778 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340781 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340784 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340786 4374 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340788 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340791 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340793 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:58.356513 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340796 4374 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340799 4374 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340801 4374 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340804 4374 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340807 4374 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340809 4374 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340812 4374 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340815 4374 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340817 4374 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340820 4374 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340822 4374 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340825 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340828 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340830 4374 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340833 4374 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.340855 4374 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 04:45:58.357108 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340898 4374 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340903 4374 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340907 4374 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340910 4374 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340913 4374 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340916 4374 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340919 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340921 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340924 4374 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340927 4374 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340930 4374 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340933 4374 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340935 4374 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340938 4374 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340941 4374 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340943 4374 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340946 4374 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340949 4374 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340951 4374 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 04:45:58.357579 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340954 4374 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340957 4374 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340959 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340962 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340964 4374 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340967 4374 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340969 4374 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340972 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340975 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340977 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340980 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340983 4374 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340985 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340988 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340993 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340996 4374 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.340998 4374 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341001 4374 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341004 4374 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 04:45:58.358083 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341007 4374 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341009 4374 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341012 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341015 4374 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341018 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341020 4374 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341023 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341026 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341028 4374 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341031 4374 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341034 4374 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341036 4374 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341039 4374 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341041 4374 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341044 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341047 4374 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341050 4374 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341052 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341055 4374 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341058 4374 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 04:45:58.358627 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341060 4374 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341063 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341065 4374 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341070 4374 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341073 4374 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341076 4374 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341078 4374 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341082 4374 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341085 4374 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341088 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341090 4374 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341093 4374 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341097 4374 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341099 4374 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341102 4374 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341105 4374 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341108 4374 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341110 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341114 4374 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 04:45:58.359150 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341117 4374 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341121 4374 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341123 4374 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341126 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341129 4374 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341131 4374 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341134 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341136 4374 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341139 4374 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341143 4374 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341146 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341149 4374 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341152 4374 feature_gate.go:386] unrecognized feature gate: Example Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341154 4374 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341157 4374 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341159 4374 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341163 4374 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341166 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341169 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 04:45:58.359614 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341189 4374 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341195 4374 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341198 4374 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341201 4374 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341204 4374 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341206 4374 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341209 4374 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341212 4374 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341216 4374 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:45:58.341219 4374 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.341266 4374 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.342223 4374 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.342276 4374 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.342451 4374 server.go:1014] "Starting client certificate rotation" Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.342587 4374 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.342642 4374 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 04:40:15 +0000 UTC" deadline="2028-05-26 05:48:10.540945251 +0000 UTC" Nov 04 04:45:58.360107 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.342668 4374 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13657h2m12.198279392s" Nov 04 04:45:58.360623 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.342913 4374 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 04:45:58.360623 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.343995 4374 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 04:45:58.360623 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.346912 4374 log.go:25] "Validated CRI v1 runtime API" Nov 04 04:45:58.360623 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.348458 4374 log.go:25] "Validated CRI v1 image API" Nov 04 04:45:58.360623 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.349166 4374 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 04:45:58.360623 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.350537 4374 fs.go:135] Filesystem UUIDs: map[5df3bbc1-7915-41d3-af3c-26d165331813:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d017aece-eb0b-47fc-8f0d-b0ec0e6736de:/dev/nvme0n1p4] Nov 04 04:45:58.360789 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.350553 4374 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/383877d45ee0186dfdd340a869fad974454cc6cf3815715a9630fe719adca3e9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/383877d45ee0186dfdd340a869fad974454cc6cf3815715a9630fe719adca3e9/userdata/shm major:0 minor:79 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/610459a2adf085c6e38b56ac3824da2ffa4455257dc5d30899cd56a26931c9a7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/610459a2adf085c6e38b56ac3824da2ffa4455257dc5d30899cd56a26931c9a7/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/66d254dc707f52bcd5471b2162264ed4d0cc8ca74201d666c3b3ec51ee45ed15/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/66d254dc707f52bcd5471b2162264ed4d0cc8ca74201d666c3b3ec51ee45ed15/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6978674a56f0167af3ec8a283d5a8fe6aefab5a734b4a8300bbac7b835634a4f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6978674a56f0167af3ec8a283d5a8fe6aefab5a734b4a8300bbac7b835634a4f/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8fd2ec5b8981239b072f71c45c2424fc32be35d27319a476aebcdfdba705a19b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8fd2ec5b8981239b072f71c45c2424fc32be35d27319a476aebcdfdba705a19b/userdata/shm major:0 minor:206 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9be3f1a7033f5f8ce3554042a3faa48702891092bf3a36b598e0e4daf81e4d0a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9be3f1a7033f5f8ce3554042a3faa48702891092bf3a36b598e0e4daf81e4d0a/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9d236d115a08f5ef6ae8733f68f9f9adbad07d48e8470e84deac192c60e053f7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9d236d115a08f5ef6ae8733f68f9f9adbad07d48e8470e84deac192c60e053f7/userdata/shm major:0 minor:203 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9d2bbdbccf9a47a129c2066af32da04db445970994c1e69c54df165fb91c4a91/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9d2bbdbccf9a47a129c2066af32da04db445970994c1e69c54df165fb91c4a91/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b152b518b2c4ab500c6c8a655d1ef9e058ad29779fd75aa42f9f654c005e0e28/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b152b518b2c4ab500c6c8a655d1ef9e058ad29779fd75aa42f9f654c005e0e28/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c19af60553a809d66f26d220a6a82e879bbad6051749947e5d56c0c1d5f62673/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c19af60553a809d66f26d220a6a82e879bbad6051749947e5d56c0c1d5f62673/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d6f34ef8de03254b9160071bb40a3e2495fd3b63b8c61b3a702f88b38efddb03/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d6f34ef8de03254b9160071bb40a3e2495fd3b63b8c61b3a702f88b38efddb03/userdata/shm major:0 minor:91 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ea6e461f4948f77f3b3fbeb64dc683a3616234616994328510a59ee35e8d938a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ea6e461f4948f77f3b3fbeb64dc683a3616234616994328510a59ee35e8d938a/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f7fbd87646d42f04b25d855a9e927485ad1afcf375b28062cd94a0165f1024fd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f7fbd87646d42f04b25d855a9e927485ad1afcf375b28062cd94a0165f1024fd/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/09c6a70b-bc11-4b06-a052-ca93a697355b/volumes/kubernetes.io~projected/kube-api-access-9v85z:{mountpoint:/var/lib/kubelet/pods/09c6a70b-bc11-4b06-a052-ca93a697355b/volumes/kubernetes.io~projected/kube-api-access-9v85z major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0/volumes/kubernetes.io~projected/kube-api-access-dnjmq:{mountpoint:/var/lib/kubelet/pods/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0/volumes/kubernetes.io~projected/kube-api-access-dnjmq major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4affa996-4479-46e2-bde8-dba1989da65d/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/4affa996-4479-46e2-bde8-dba1989da65d/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4affa996-4479-46e2-bde8-dba1989da65d/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/4affa996-4479-46e2-bde8-dba1989da65d/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4affa996-4479-46e2-bde8-dba1989da65d/volumes/kubernetes.io~projected/kube-api-access-tstz6:{mountpoint:/var/lib/kubelet/pods/4affa996-4479-46e2-bde8-dba1989da65d/volumes/kubernetes.io~projected/kube-api-access-tstz6 major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/53eaa748-5fc9-4d33-8fd4-946c97ec0074/volumes/kubernetes.io~projected/kube-api-access-dgt87:{mountpoint:/var/lib/kubelet/pods/53eaa748-5fc9-4d33-8fd4-946c97ec0074/volumes/kubernetes.io~projected/kube-api-access-dgt87 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/53eaa748-5fc9-4d33-8fd4-946c97ec0074/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/53eaa748-5fc9-4d33-8fd4-946c97ec0074/volumes/kubernetes.io~secret/metrics-certs major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/58c2628d-e8f4-4dd4-987e-c7c4f3847fac/volumes/kubernetes.io~projected/kube-api-access-hr56b:{mountpoint:/var/lib/kubelet/pods/58c2628d-e8f4-4dd4-987e-c7c4f3847fac/volumes/kubernetes.io~projected/kube-api-access-hr56b major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ad2cc585-2e96-43fe-b778-f19bb57ac50d/volumes/kubernetes.io~projected/kube-api-access-4nbph:{mountpoint:/var/lib/kubelet/pods/ad2cc585-2e96-43fe-b778-f19bb57ac50d/volumes/kubernetes.io~projected/kube-api-access-4nbph major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b2224b84-c10c-46e8-b7ba-844bc5802f34/volumes/kubernetes.io~projected/kube-api-access-bfb82:{mountpoint:/var/lib/kubelet/pods/b2224b84-c10c-46e8-b7ba-844bc5802f34/volumes/kubernetes.io~projected/kube-api-access-bfb82 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b94ca77d-354d-4404-99a9-dd6dada01c24/volumes/kubernetes.io~projected/kube-api-access-sf6dc:{mountpoint:/var/lib/kubelet/pods/b94ca77d-354d-4404-99a9-dd6dada01c24/volumes/kubernetes.io~projected/kube-api-access-sf6dc major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c/volumes/kubernetes.io~projected/kube-api-access-knj8p:{mountpoint:/var/lib/kubelet/pods/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c/volumes/kubernetes.io~projected/kube-api-access-knj8p major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d282c492-0fef-42cf-922e-86b1355221f2/volumes/kubernetes.io~projected/kube-api-access-s778k:{mountpoint:/var/lib/kubelet/pods/d282c492-0fef-42cf-922e-86b1355221f2/volumes/kubernetes.io~projected/kube-api-access-s778k major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ddd38197-2fcb-4086-a376-b5ef4628be3c/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/ddd38197-2fcb-4086-a376-b5ef4628be3c/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed101df4-83a6-42e0-8638-8b99890a8553/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/ed101df4-83a6-42e0-8638-8b99890a8553/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/175f0c589785bf068bffb9d6ff6a45daa4d03bb0222511f94edf0cd44fde9a11/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/d7d46480ea5c9bc03cd03782bb4d0d480fd974f2f7f41513bf457bdce8da4108/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/0edc84629a47882deab4880925b7d570d2a0bf1215edb30f7c9c3b56fd214af3/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/8fbf8caf0041a51d03d54b5c97b4baa38fef0c0ce361742d2407640f883ef89d/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-112:{mountpoint:/var/lib/containers/storage/overlay/b30344fe15545f5e6639c33d687a1b872398b548a5c10692a03841187bb4c4e0/merged major:0 minor:112 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/c4532644b9f97a5b05091495e030f4e1f6004270d39ea95ec3c20021d00014de/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/a5590c9cfd67b9961d097ff39601125d35ca7f67e0ee5f3941b2835f6ca4de74/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/d63407f44111c339954a666e47cef0941f7227c9b53dc6959f47f2898156b18c/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/0d553f3949383e5108e22db05a59c9e9ecddbfb30c2d79f2dbce4305a8c78378/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/7a41ef935dad5662b3eb902cb01d8b09c0bff9383ada1dfd5adbb7d7636c5e93/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/53229aebc21d30f3ee23b0715726c2b56d431659bb80bbb62e5a5879e112e4f0/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/abb0473b193572f572a9f8dce49cd882e28b2fd24adbf936ffc4235733350f0d/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/b87865a7bcc1c48802bd1f3a24927f14f21b0363536bccea638d8e208c35b351/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/34b0d4112be1eb6602d91533104de257906b2303f05e309b449272a2364808d6/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/9c6902277e9b4bcb05be0200101aed12d90a55d36f0d7238190c1ecb4184dae9/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/1ed5d486afd088866c96add80f949da3e61ea889514ec7d67dd30393b114c3a2/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/a434f65799b1451d59afc898fc950406dd7a3082f46338fbbec8d0a6e65cbf92/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d75ea854f609120e5dc2909a974c56db9fb3dcc0f6c16e62e26c7a219b28dd7f/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/2613a31261d5ec2524bab155f3cb8256d0a3b1db656e268c5ea3a71d7f16b0fb/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/1e8d11364f700dfddf2f38008f5297022b5aee824e8490a3a02012ccf1c10a18/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/df7dfe1e853731e2095e1429d5ed74944de3f3761d62be637024d3fb990e2d4b/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-208:{mountpoint:/var/lib/containers/storage/overlay/5a83efbae771b8d24a2dd3695471f302dbdc741298351f3686738c4740558b3b/merged major:0 minor:208 fsType:overlay blockSize:0} overlay_0-210:{mountpoint:/var/lib/containers/storage/overlay/f9868078a7cc4e3f07c1df44fa2a564a58023b8ad41a7eac22a7bc5ec2c9802e/merged major:0 minor:210 fsType:overlay blockSize:0} overlay_0-212:{mountpoint:/var/lib/containers/storage/overlay/72da46655ef2b72002c30c101a9741a2c69b0bf626cf64848aa4237e9c9ca219/merged major:0 minor:212 fsType:overlay blockSize:0} overlay_0-214:{mountpoint:/var/lib/containers/storage/overlay/554ac7e52a7e8a0e13f1dcd02c448350e402c17578971d55598282a5e9cce791/merged major:0 minor:214 fsType:overlay blockSize:0} overlay_0-216:{mountpoint:/var/lib/containers/storage/overlay/d4da0ba73f98362740ec662cad55a79996678d3b2386a9aa1d175e6fbfd895b3/merged major:0 minor:216 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/dc2f7748babc13fe9c83d00747ff8f1cd01c78af0e641210d0dd681263eb3fa3/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-227:{mountpoint:/var/lib/containers/storage/overlay/48dbc6e4d81843fb9c799d84843bb904f2185d4261718c1d2f28899ca651ebfc/merged major:0 minor:227 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/94104857c268ccd2c82824b8ad82cf86bebd27dd048997eadbd0b52deca2cbe0/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/7ba9e3b624dcdcf3862fb030f5efbac626d4c31dbd95cdf663afbfb5f40c0749/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/a497572d5f8e497e1aeb429d52da1364640fea68fc2401bc6e5a1cd82605b674/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/62c4b492c702fd85c112325793de4fbb337fec3d50b89fb7fdca3464c5c6c475/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/f785725fa843f386c6d380b485d45ac19eba2e00b91ae1cc8258ee3475ea39c9/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-81:{mountpoint:/var/lib/containers/storage/overlay/247a85ad3bb591abb350a99099d1956a2f9550e7a2dc55e4f6ae28601b2d5b41/merged major:0 minor:81 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/b5e6f505bb107e40277b61937cbc9aa287ea6f6bceabf9e84c93c61447fadf2e/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/a6bd774c69b99b617f871910bd4e8db715e6cf26d04366e3386b5fe9580fdb46/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/9c2c7c4846d754a2c369bb6e38407bd6bb1e51a7080f7bbf399c16681eea15a3/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/933ed219e46815bf6b0de9f155990f33916be85c16e17d45c8353c5985355668/merged major:0 minor:96 fsType:overlay blockSize:0}] Nov 04 04:45:58.360789 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.356099 4374 manager.go:217] Machine: {Timestamp:2026-11-04 04:45:58.35562674 +0000 UTC m=+0.067317871 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100965 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a2a5c37c7e29e790d8c742a9b33d2 SystemUUID:ec2a2a5c-37c7-e29e-790d-8c742a9b33d2 BootID:a7603a86-74f2-49af-b6ed-cf666b70ba29 Filesystems:[{Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9d236d115a08f5ef6ae8733f68f9f9adbad07d48e8470e84deac192c60e053f7/userdata/shm DeviceMajor:0 DeviceMinor:203 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9d2bbdbccf9a47a129c2066af32da04db445970994c1e69c54df165fb91c4a91/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/53eaa748-5fc9-4d33-8fd4-946c97ec0074/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c/volumes/kubernetes.io~projected/kube-api-access-knj8p DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/66d254dc707f52bcd5471b2162264ed4d0cc8ca74201d666c3b3ec51ee45ed15/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f7fbd87646d42f04b25d855a9e927485ad1afcf375b28062cd94a0165f1024fd/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/09c6a70b-bc11-4b06-a052-ca93a697355b/volumes/kubernetes.io~projected/kube-api-access-9v85z DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-210 DeviceMajor:0 DeviceMinor:210 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/4affa996-4479-46e2-bde8-dba1989da65d/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-227 DeviceMajor:0 DeviceMinor:227 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-81 DeviceMajor:0 DeviceMinor:81 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/610459a2adf085c6e38b56ac3824da2ffa4455257dc5d30899cd56a26931c9a7/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/53eaa748-5fc9-4d33-8fd4-946c97ec0074/volumes/kubernetes.io~projected/kube-api-access-dgt87 DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0/volumes/kubernetes.io~projected/kube-api-access-dnjmq DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-212 DeviceMajor:0 DeviceMinor:212 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ed101df4-83a6-42e0-8638-8b99890a8553/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ad2cc585-2e96-43fe-b778-f19bb57ac50d/volumes/kubernetes.io~projected/kube-api-access-4nbph DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6978674a56f0167af3ec8a283d5a8fe6aefab5a734b4a8300bbac7b835634a4f/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8fd2ec5b8981239b072f71c45c2424fc32be35d27319a476aebcdfdba705a19b/userdata/shm DeviceMajor:0 DeviceMinor:206 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4affa996-4479-46e2-bde8-dba1989da65d/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d282c492-0fef-42cf-922e-86b1355221f2/volumes/kubernetes.io~projected/kube-api-access-s778k DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b2224b84-c10c-46e8-b7ba-844bc5802f34/volumes/kubernetes.io~projected/kube-api-access-bfb82 DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-112 DeviceMajor:0 DeviceMinor:112 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4affa996-4479-46e2-bde8-dba1989da65d/volumes/kubernetes.io~projected/kube-api-access-tstz6 DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-208 DeviceMajor:0 DeviceMinor:208 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9be3f1a7033f5f8ce3554042a3faa48702891092bf3a36b598e0e4daf81e4d0a/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d6f34ef8de03254b9160071bb40a3e2495fd3b63b8c61b3a702f88b38efddb03/userdata/shm DeviceMajor:0 DeviceMinor:91 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ea6e461f4948f77f3b3fbeb64dc683a3616234616994328510a59ee35e8d938a/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-216 DeviceMajor:0 DeviceMinor:216 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/58c2628d-e8f4-4dd4-987e-c7c4f3847fac/volumes/kubernetes.io~projected/kube-api-access-hr56b DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b152b518b2c4ab500c6c8a655d1ef9e058ad29779fd75aa42f9f654c005e0e28/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-214 DeviceMajor:0 DeviceMinor:214 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b94ca77d-354d-4404-99a9-dd6dada01c24/volumes/kubernetes.io~projected/kube-api-access-sf6dc DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ddd38197-2fcb-4086-a376-b5ef4628be3c/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c19af60553a809d66f26d220a6a82e879bbad6051749947e5d56c0c1d5f62673/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/383877d45ee0186dfdd340a869fad974454cc6cf3815715a9630fe719adca3e9/userdata/shm DeviceMajor:0 DeviceMinor:79 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:8fd2ec5b8981239 MacAddress:da:94:9f:4e:02:2c Speed:10000 Mtu:8901} {Name:9d236d115a08f5e MacAddress:fa:56:bd:7f:09:8b Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:c9:ea:95:c1 Speed:0 Mtu:9001} {Name:br-int MacAddress:8e:37:3f:fb:2a:b4 Speed:0 Mtu:8901} {Name:ea6e461f4948f77 MacAddress:ca:b5:ae:ef:e9:c3 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:c9:ea:95:c1 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:16:4e:12:6a:6f:d0 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:16:61:e0:fd:6c:9e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 04:45:58.360789 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.356317 4374 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 04:45:58.360789 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.356364 4374 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 04:45:58.360789 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.356622 4374 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.356642 4374 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-242.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.356756 4374 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.356765 4374 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.356778 4374 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.356790 4374 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.356881 4374 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.357027 4374 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.357129 4374 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.357163 4374 kubelet.go:493] "Attempting to sync node with API server" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.357180 4374 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.357196 4374 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.357208 4374 kubelet.go:405] "Adding apiserver pod source" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.357222 4374 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.357924 4374 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.357934 4374 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.358348 4374 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.360857 4374 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.360884 4374 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361007 4374 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Nov 04 04:45:58.361245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361244 4374 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361273 4374 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361375 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361387 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361393 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361400 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361406 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361415 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361424 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361431 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361439 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361445 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361453 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361463 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361487 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 04:45:58.361850 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.361497 4374 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 04:45:58.364725 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.364709 4374 server.go:1263] "Started kubelet" Nov 04 04:45:58.365576 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.364806 4374 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 04:45:58.365576 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.364804 4374 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 04:45:58.365576 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.365401 4374 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 04:45:58.365479 ip-10-0-140-242 systemd[1]: Started Kubernetes Kubelet. Nov 04 04:45:58.369099 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.366975 4374 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 04:45:58.369099 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.368903 4374 server.go:317] "Adding debug handlers to kubelet server" Nov 04 04:45:58.369595 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.369574 4374 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 04:45:58.370993 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.370491 4374 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 04:40:17 +0000 UTC" deadline="2028-05-20 22:25:52.148074898 +0000 UTC" Nov 04 04:45:58.370993 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.370536 4374 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13529h39m53.777542072s" Nov 04 04:45:58.370993 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.370554 4374 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 04:45:58.370993 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.370801 4374 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 04:45:58.370993 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.370814 4374 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 04:45:58.370993 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.370917 4374 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 04:45:58.371325 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:45:58.371018 4374 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:58.371961 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.371914 4374 factory.go:153] Registering CRI-O factory Nov 04 04:45:58.371961 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.371932 4374 factory.go:223] Registration of the crio container factory successfully Nov 04 04:45:58.372141 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.371981 4374 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 04:45:58.372141 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.371990 4374 factory.go:55] Registering systemd factory Nov 04 04:45:58.372141 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.371999 4374 factory.go:223] Registration of the systemd container factory successfully Nov 04 04:45:58.372141 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.372013 4374 factory.go:103] Registering Raw factory Nov 04 04:45:58.372141 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.372027 4374 manager.go:1199] Started watching for new ooms in manager Nov 04 04:45:58.372378 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.372336 4374 manager.go:319] Starting recovery of all containers Nov 04 04:45:58.374118 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374081 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd6b4c99-03f6-42fb-9a97-ff51e2efb90c" volumeName="kubernetes.io/projected/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-kube-api-access-knj8p" seLinuxMountContext="" Nov 04 04:45:58.374217 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374117 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ddd38197-2fcb-4086-a376-b5ef4628be3c" volumeName="kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret" seLinuxMountContext="" Nov 04 04:45:58.374217 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374156 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed101df4-83a6-42e0-8638-8b99890a8553" volumeName="kubernetes.io/secret/ed101df4-83a6-42e0-8638-8b99890a8553-agent-certs" seLinuxMountContext="" Nov 04 04:45:58.374217 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374170 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09c6a70b-bc11-4b06-a052-ca93a697355b" volumeName="kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z" seLinuxMountContext="" Nov 04 04:45:58.374217 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374185 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" volumeName="kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-whereabouts-flatfile-configmap" seLinuxMountContext="" Nov 04 04:45:58.374217 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374211 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" volumeName="kubernetes.io/projected/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-kube-api-access-dnjmq" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374225 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4affa996-4479-46e2-bde8-dba1989da65d" volumeName="kubernetes.io/projected/4affa996-4479-46e2-bde8-dba1989da65d-kube-api-access-tstz6" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374237 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="58c2628d-e8f4-4dd4-987e-c7c4f3847fac" volumeName="kubernetes.io/configmap/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-serviceca" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374246 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b94ca77d-354d-4404-99a9-dd6dada01c24" volumeName="kubernetes.io/empty-dir/b94ca77d-354d-4404-99a9-dd6dada01c24-tmp-dir" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374254 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd6b4c99-03f6-42fb-9a97-ff51e2efb90c" volumeName="kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovnkube-script-lib" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374262 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd6b4c99-03f6-42fb-9a97-ff51e2efb90c" volumeName="kubernetes.io/secret/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovn-node-metrics-cert" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374268 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" volumeName="kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cni-sysctl-allowlist" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374276 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4affa996-4479-46e2-bde8-dba1989da65d" volumeName="kubernetes.io/empty-dir/4affa996-4479-46e2-bde8-dba1989da65d-tmp" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374291 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4affa996-4479-46e2-bde8-dba1989da65d" volumeName="kubernetes.io/empty-dir/4affa996-4479-46e2-bde8-dba1989da65d-etc-tuned" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374306 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53eaa748-5fc9-4d33-8fd4-946c97ec0074" volumeName="kubernetes.io/projected/53eaa748-5fc9-4d33-8fd4-946c97ec0074-kube-api-access-dgt87" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374319 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd6b4c99-03f6-42fb-9a97-ff51e2efb90c" volumeName="kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovnkube-config" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374332 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d282c492-0fef-42cf-922e-86b1355221f2" volumeName="kubernetes.io/projected/d282c492-0fef-42cf-922e-86b1355221f2-kube-api-access-s778k" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374344 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad2cc585-2e96-43fe-b778-f19bb57ac50d" volumeName="kubernetes.io/configmap/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-daemon-config" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374356 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad2cc585-2e96-43fe-b778-f19bb57ac50d" volumeName="kubernetes.io/configmap/ad2cc585-2e96-43fe-b778-f19bb57ac50d-cni-binary-copy" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374367 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad2cc585-2e96-43fe-b778-f19bb57ac50d" volumeName="kubernetes.io/projected/ad2cc585-2e96-43fe-b778-f19bb57ac50d-kube-api-access-4nbph" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374378 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2224b84-c10c-46e8-b7ba-844bc5802f34" volumeName="kubernetes.io/projected/b2224b84-c10c-46e8-b7ba-844bc5802f34-kube-api-access-bfb82" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374389 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b94ca77d-354d-4404-99a9-dd6dada01c24" volumeName="kubernetes.io/projected/b94ca77d-354d-4404-99a9-dd6dada01c24-kube-api-access-sf6dc" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374401 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed101df4-83a6-42e0-8638-8b99890a8553" volumeName="kubernetes.io/configmap/ed101df4-83a6-42e0-8638-8b99890a8553-konnectivity-ca" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374431 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" volumeName="kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cni-binary-copy" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374445 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53eaa748-5fc9-4d33-8fd4-946c97ec0074" volumeName="kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374456 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="58c2628d-e8f4-4dd4-987e-c7c4f3847fac" volumeName="kubernetes.io/projected/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-kube-api-access-hr56b" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374468 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2224b84-c10c-46e8-b7ba-844bc5802f34" volumeName="kubernetes.io/configmap/b2224b84-c10c-46e8-b7ba-844bc5802f34-iptables-alerter-script" seLinuxMountContext="" Nov 04 04:45:58.374472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374482 4374 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd6b4c99-03f6-42fb-9a97-ff51e2efb90c" volumeName="kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-env-overrides" seLinuxMountContext="" Nov 04 04:45:58.375352 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374495 4374 reconstruct.go:97] "Volume reconstruction finished" Nov 04 04:45:58.375352 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.374502 4374 reconciler.go:29] "Reconciler: start to sync state" Nov 04 04:45:58.402983 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.402924 4374 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 04:45:58.404277 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.404251 4374 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 04:45:58.404377 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.404280 4374 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 04:45:58.404377 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.404303 4374 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 04:45:58.404377 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:45:58.404348 4374 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Nov 04 04:45:58.417033 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.417008 4374 generic.go:356] "Generic (PLEG): container finished" podID="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" containerID="4a0d9396befb3530a130c0c8cb28b1595d4a40e17308863d9c865e3b2a42e1b4" exitCode=0 Nov 04 04:45:58.417033 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.417029 4374 generic.go:356] "Generic (PLEG): container finished" podID="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" containerID="6ee384c6c84bc2eae9d72dfecab4914909f3541d2f5ecca54773c4a35a6f0a54" exitCode=0 Nov 04 04:45:58.417208 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.417039 4374 generic.go:356] "Generic (PLEG): container finished" podID="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" containerID="66162b4fee7f656de1905d6a7715e2f361b08ae219cb9a86de66c9d3dc8c8431" exitCode=0 Nov 04 04:45:58.417208 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.417051 4374 generic.go:356] "Generic (PLEG): container finished" podID="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" containerID="0a371c35877d97b12124a2510b733eb4940f0c1a8dffcf2d9ba76d11f997eee7" exitCode=0 Nov 04 04:45:58.417208 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.417058 4374 generic.go:356] "Generic (PLEG): container finished" podID="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" containerID="dcf2f25d282df7d33e799b70792fcb4cbbd6fb842f355f819fcebc1720e0b688" exitCode=0 Nov 04 04:45:58.417208 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.417063 4374 generic.go:356] "Generic (PLEG): container finished" podID="456c901b-f6fa-4de0-85aa-59d4fc1ec5c0" containerID="cb112106c49102d35c0fd31620ed7934c1a535f2e75294a6e0c96484da340322" exitCode=0 Nov 04 04:45:58.426359 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.426337 4374 generic.go:356] "Generic (PLEG): container finished" podID="c7105a6f9939c228417dda3a14245b0d" containerID="c3c1ce9b24df4e674e18b39b34ff88edbbe188e6e5016a586ce2c066f54e9363" exitCode=0 Nov 04 04:45:58.436629 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.436613 4374 manager.go:324] Recovery completed Nov 04 04:45:58.448689 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.448672 4374 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:58.449842 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.449828 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:58.449914 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.449858 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:58.449914 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.449891 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:58.451121 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451104 4374 cpu_manager.go:226] "Starting" policy="none" Nov 04 04:45:58.451121 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451116 4374 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 04:45:58.451121 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451119 4374 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:58.451300 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451131 4374 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 04:45:58.451300 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451236 4374 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Nov 04 04:45:58.451300 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451246 4374 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Nov 04 04:45:58.451300 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451261 4374 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Nov 04 04:45:58.451300 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451267 4374 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Nov 04 04:45:58.451300 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451272 4374 policy_none.go:50] "Start" Nov 04 04:45:58.451300 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451280 4374 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 04:45:58.451300 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451291 4374 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 04:45:58.451549 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451408 4374 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Nov 04 04:45:58.451549 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451414 4374 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Nov 04 04:45:58.451549 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.451420 4374 policy_none.go:44] "Start" Nov 04 04:45:58.452024 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.452006 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:58.452089 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.452032 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:58.452089 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.452043 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:58.455220 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.455188 4374 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 04:45:58.459687 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.459671 4374 manager.go:341] "Starting Device Plugin manager" Nov 04 04:45:58.459763 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:45:58.459698 4374 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 04:45:58.459763 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.459708 4374 server.go:87] "Starting device plugin registration server" Nov 04 04:45:58.459954 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.459940 4374 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 04:45:58.459997 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.459972 4374 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 04:45:58.460043 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.460022 4374 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 04:45:58.460150 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.460109 4374 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 04:45:58.460262 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.460155 4374 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 04:45:58.460262 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.460161 4374 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 04:45:58.460262 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.460195 4374 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T04:45:58.460145182Z","UUID":"697a6216-2a40-4d04-b9fa-f9ce41cb4949","Handler":null,"Name":"","Endpoint":""} Nov 04 04:45:58.461298 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:45:58.461279 4374 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 04:45:58.461380 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:45:58.461317 4374 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" Nov 04 04:45:58.461508 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.461490 4374 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 04:45:58.461569 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.461513 4374 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 04:45:58.505854 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.505431 4374 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal"] Nov 04 04:45:58.505967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.505954 4374 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:58.506994 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.506982 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:58.507043 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.507008 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:58.507043 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.507024 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:58.507218 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.507208 4374 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:58.507357 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.507342 4374 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:58.508015 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.508000 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:58.508015 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.508011 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:58.508147 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.508025 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:58.508147 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.508035 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:58.508147 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.508050 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:58.508147 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.508037 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:58.508327 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:45:58.508229 4374 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" node="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.508391 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.508319 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" event={"ID":"c7105a6f9939c228417dda3a14245b0d","Type":"ContainerStarted","Data":"96ddf95e63cc4c6abb70d34a2d161a91fe8902f7cb298d93716d8f1afa12b5bf"} Nov 04 04:45:58.508391 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.508371 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" event={"ID":"c7105a6f9939c228417dda3a14245b0d","Type":"ContainerDied","Data":"c3c1ce9b24df4e674e18b39b34ff88edbbe188e6e5016a586ce2c066f54e9363"} Nov 04 04:45:58.508391 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.508380 4374 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:58.508522 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.508391 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" event={"ID":"c7105a6f9939c228417dda3a14245b0d","Type":"ContainerStarted","Data":"c19af60553a809d66f26d220a6a82e879bbad6051749947e5d56c0c1d5f62673"} Nov 04 04:45:58.508522 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.508444 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" event={"ID":"85dfaf93b35e741480ab992c223ae102","Type":"ContainerStarted","Data":"cb403ef0eeb644a1e803e1b994eb249f9b51cd4b87bc9fe437f8517de4810220"} Nov 04 04:45:58.508522 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.508463 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" event={"ID":"85dfaf93b35e741480ab992c223ae102","Type":"ContainerStarted","Data":"9be3f1a7033f5f8ce3554042a3faa48702891092bf3a36b598e0e4daf81e4d0a"} Nov 04 04:45:58.509248 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.509234 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:58.509315 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.509262 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:58.509315 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.509276 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:58.509430 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:45:58.509415 4374 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-242.us-west-2.compute.internal\" not found" node="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.560620 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.560595 4374 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 04:45:58.560791 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.560775 4374 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 04:45:58.560895 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.560814 4374 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Nov 04 04:45:58.561585 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.561571 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 04:45:58.561622 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.561598 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 04:45:58.561622 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.561608 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 04:45:58.561675 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.561635 4374 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.604908 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.604886 4374 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 04:45:58.605009 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.604996 4374 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 04:45:58.607460 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.607432 4374 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 04:45:58.661252 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.661227 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c7105a6f9939c228417dda3a14245b0d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"c7105a6f9939c228417dda3a14245b0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.661339 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.661261 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7105a6f9939c228417dda3a14245b0d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"c7105a6f9939c228417dda3a14245b0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.661339 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.661288 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85dfaf93b35e741480ab992c223ae102-config\") pod \"kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"85dfaf93b35e741480ab992c223ae102\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.671507 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.671486 4374 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.699819 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.699802 4374 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.699863 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.699858 4374 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.700389 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.700373 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeNotReady" Nov 04 04:45:58.700459 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.700400 4374 setters.go:546] "Node became not ready" node="ip-10-0-140-242.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-04T04:45:58Z","lastTransitionTime":"2026-11-04T04:45:58Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"} Nov 04 04:45:58.715677 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.715658 4374 nodeinfomanager.go:471] existing CSINode "ip-10-0-140-242.us-west-2.compute.internal" is owned by different node (oldNodeID="ed7f1341-1251-4069-8634-a81891b9f3d3", newNodeID=""), cleaning up... Nov 04 04:45:58.727964 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.727946 4374 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 04:45:58.728037 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:45:58.727993 4374 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.728037 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.728005 4374 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.761880 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.761778 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c7105a6f9939c228417dda3a14245b0d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"c7105a6f9939c228417dda3a14245b0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.761880 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.761843 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7105a6f9939c228417dda3a14245b0d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"c7105a6f9939c228417dda3a14245b0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.762031 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.761887 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85dfaf93b35e741480ab992c223ae102-config\") pod \"kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"85dfaf93b35e741480ab992c223ae102\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.762031 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.761903 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c7105a6f9939c228417dda3a14245b0d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"c7105a6f9939c228417dda3a14245b0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.762031 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.761935 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85dfaf93b35e741480ab992c223ae102-config\") pod \"kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"85dfaf93b35e741480ab992c223ae102\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.762031 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.761961 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7105a6f9939c228417dda3a14245b0d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" (UID: \"c7105a6f9939c228417dda3a14245b0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:58.785163 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:58.785138 4374 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 04:45:58.785270 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:45:58.785184 4374 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:59.357956 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.357915 4374 apiserver.go:52] "Watching apiserver" Nov 04 04:45:59.442344 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.442319 4374 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:59.442471 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.442448 4374 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:59.454424 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.454402 4374 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 04:45:59.454776 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.454758 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-stlg9","openshift-image-registry/node-ca-n278n","openshift-ovn-kubernetes/ovnkube-node-j7pdj","openshift-dns/node-resolver-b4n8j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal","openshift-network-diagnostics/network-check-target-z99g5","kube-system/konnectivity-agent-8zd4x","openshift-multus/multus-additional-cni-plugins-mvpj9","openshift-multus/multus-lfwvp","kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v","openshift-cluster-node-tuning-operator/tuned-jkgz4","openshift-multus/network-metrics-daemon-m87zm","openshift-network-operator/iptables-alerter-7n7vn"] Nov 04 04:45:59.472493 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.472474 4374 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 04:45:59.499688 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.499669 4374 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 04:45:59.499783 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:45:59.499713 4374 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:59.502321 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.502307 4374 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 04:45:59.502367 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:45:59.502340 4374 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-242.us-west-2.compute.internal" Nov 04 04:45:59.507915 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.507894 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nhf59\"" Nov 04 04:45:59.508040 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.507903 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-95fm7\"" Nov 04 04:45:59.508040 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.507916 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 04:45:59.508040 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.507916 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-gwbb5\"" Nov 04 04:45:59.508040 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.507950 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 04:45:59.508245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.507977 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ss4dp\"" Nov 04 04:45:59.508245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508035 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 04:45:59.508245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508084 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 04:45:59.508245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508186 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 04:45:59.508245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508189 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 04:45:59.508245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508210 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kjpq6\"" Nov 04 04:45:59.508245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508225 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 04:45:59.508245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508228 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 04:45:59.508245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508236 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 04:45:59.508556 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508255 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 04:45:59.508556 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508261 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 04:45:59.508556 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508278 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 04:45:59.508556 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508277 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 04:45:59.508556 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508291 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 04:45:59.508974 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.508955 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8xbd4\"" Nov 04 04:45:59.509102 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.509078 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 04:45:59.509218 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.509115 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 04:45:59.509218 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.509161 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m97m8\"" Nov 04 04:45:59.509314 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.509258 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 04:45:59.509314 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.509117 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 04:45:59.509409 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.509349 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7qqph\"" Nov 04 04:45:59.509559 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.509543 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-77ccl\"" Nov 04 04:45:59.509749 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.509731 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-rrdk2\"" Nov 04 04:45:59.509818 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.509791 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-45k46\"" Nov 04 04:45:59.509894 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.509860 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 04:45:59.510025 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.510008 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 04:45:59.511598 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.511570 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 04:45:59.566849 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.566818 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovn-node-metrics-cert\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.566849 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.566849 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf6dc\" (UniqueName: \"kubernetes.io/projected/b94ca77d-354d-4404-99a9-dd6dada01c24-kube-api-access-sf6dc\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:59.567053 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.566879 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-system-cni-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.567053 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.566896 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-netns\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.567053 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.566912 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ddd38197-2fcb-4086-a376-b5ef4628be3c-dbus\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:59.567053 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567021 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-registration-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.567053 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567048 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr56b\" (UniqueName: \"kubernetes.io/projected/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-kube-api-access-hr56b\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:59.567267 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567073 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b2224b84-c10c-46e8-b7ba-844bc5802f34-iptables-alerter-script\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:59.567267 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567135 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.567267 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567184 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad2cc585-2e96-43fe-b778-f19bb57ac50d-cni-binary-copy\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.567267 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567227 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ddd38197-2fcb-4086-a376-b5ef4628be3c-kubelet-config\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:59.567406 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567268 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s778k\" (UniqueName: \"kubernetes.io/projected/d282c492-0fef-42cf-922e-86b1355221f2-kube-api-access-s778k\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.567406 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567299 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-run-netns\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.567406 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567326 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:59.567406 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567333 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovn-node-metrics-cert\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.567406 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567367 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.567570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567447 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnjmq\" (UniqueName: \"kubernetes.io/projected/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-kube-api-access-dnjmq\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.567570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567459 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b2224b84-c10c-46e8-b7ba-844bc5802f34-iptables-alerter-script\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:59.567570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567484 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-etc-kubernetes\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.567570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567513 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-etc-selinux\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.567570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567538 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53eaa748-5fc9-4d33-8fd4-946c97ec0074-metrics-certs\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:59.567570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567546 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-sys-fs\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.567570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567572 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-systemd-units\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.567803 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567592 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-system-cni-dir\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.567803 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567613 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cnibin\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.567803 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567646 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysconfig\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.567803 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567672 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysctl-d\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.567803 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567687 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-socket-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.567803 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567714 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovnkube-script-lib\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.567803 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567735 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v85z\" (UniqueName: \"kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z\") pod \"network-check-target-z99g5\" (UID: \"09c6a70b-bc11-4b06-a052-ca93a697355b\") " pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:59.567803 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567779 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-sys\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.568082 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567809 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-var-lib-kubelet\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.568082 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567832 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgt87\" (UniqueName: \"kubernetes.io/projected/53eaa748-5fc9-4d33-8fd4-946c97ec0074-kube-api-access-dgt87\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:59.568082 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567848 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-lib-modules\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.568082 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567892 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4affa996-4479-46e2-bde8-dba1989da65d-etc-tuned\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.568082 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567937 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2224b84-c10c-46e8-b7ba-844bc5802f34-host-slash\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:59.568082 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567955 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovnkube-script-lib\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.568082 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.567974 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4affa996-4479-46e2-bde8-dba1989da65d-etc-tuned\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.568082 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568028 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-cni-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.568082 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568052 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-kubelet\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.568082 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568083 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-conf-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.568430 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568104 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-serviceca\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:59.568430 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568128 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysctl-conf\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.568430 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568188 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b94ca77d-354d-4404-99a9-dd6dada01c24-tmp-dir\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:59.568430 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568211 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nbph\" (UniqueName: \"kubernetes.io/projected/ad2cc585-2e96-43fe-b778-f19bb57ac50d-kube-api-access-4nbph\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.568430 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568247 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b94ca77d-354d-4404-99a9-dd6dada01c24-tmp-dir\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:59.568430 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568227 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-cni-bin\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.568430 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568355 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.568430 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568395 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-socket-dir-parent\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.568701 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568442 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.568701 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568481 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-device-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.568701 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568511 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-kubelet\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.568701 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568569 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-var-lib-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.568701 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568586 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-systemd\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.568701 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568603 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-k8s-cni-cncf-io\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.568701 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568627 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.568701 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568661 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.568967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568710 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-kubernetes\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.568967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568740 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-cni-bin\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.568967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568765 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-serviceca\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:59.568967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568789 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:59.568967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568816 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-ovn\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.568967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568892 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.568967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568909 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-cni-multus\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.568967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568926 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-daemon-config\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.568967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568944 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-etc-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.569246 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.568972 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-cni-netd\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.569246 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569048 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-env-overrides\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.569246 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569067 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tstz6\" (UniqueName: \"kubernetes.io/projected/4affa996-4479-46e2-bde8-dba1989da65d-kube-api-access-tstz6\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.569246 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569111 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-cnibin\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.569246 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569154 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-hostroot\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.569246 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569166 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-daemon-config\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.569246 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569178 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-systemd\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.569246 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569220 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-run-ovn-kubernetes\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.569561 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569259 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knj8p\" (UniqueName: \"kubernetes.io/projected/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-kube-api-access-knj8p\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.569561 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569293 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-run\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.569561 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569334 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-multus-certs\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.569561 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569364 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-log-socket\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.569561 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569387 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-modprobe-d\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.569561 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569437 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-host\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.569561 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569460 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed101df4-83a6-42e0-8638-8b99890a8553-agent-certs\") pod \"konnectivity-agent-8zd4x\" (UID: \"ed101df4-83a6-42e0-8638-8b99890a8553\") " pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:59.569561 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569489 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed101df4-83a6-42e0-8638-8b99890a8553-konnectivity-ca\") pod \"konnectivity-agent-8zd4x\" (UID: \"ed101df4-83a6-42e0-8638-8b99890a8553\") " pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:59.569561 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569513 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-slash\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.569561 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569553 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-node-log\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.569845 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569650 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-host\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:59.569845 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569667 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4affa996-4479-46e2-bde8-dba1989da65d-tmp\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.569845 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569690 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed101df4-83a6-42e0-8638-8b99890a8553-konnectivity-ca\") pod \"konnectivity-agent-8zd4x\" (UID: \"ed101df4-83a6-42e0-8638-8b99890a8553\") " pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:59.569845 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569696 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed101df4-83a6-42e0-8638-8b99890a8553-agent-certs\") pod \"konnectivity-agent-8zd4x\" (UID: \"ed101df4-83a6-42e0-8638-8b99890a8553\") " pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:45:59.569845 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569716 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b94ca77d-354d-4404-99a9-dd6dada01c24-hosts-file\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:59.569845 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569756 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4affa996-4479-46e2-bde8-dba1989da65d-tmp\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.569845 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569778 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfb82\" (UniqueName: \"kubernetes.io/projected/b2224b84-c10c-46e8-b7ba-844bc5802f34-kube-api-access-bfb82\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:59.569845 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.569805 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovnkube-config\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.570110 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.570002 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-ovnkube-config\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.607508 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.607486 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 04:45:59.607508 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.607502 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 04:45:59.607698 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.607508 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 04:45:59.607698 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.607540 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 04:45:59.607698 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.607542 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 04:45:59.607698 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.607582 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 04:45:59.608047 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.607999 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-l2xwd\"" Nov 04 04:45:59.608047 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.608031 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 04:45:59.608047 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.608031 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 04:45:59.608047 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.608045 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 04:45:59.608218 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.608078 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 04:45:59.609327 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.609309 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-env-overrides\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.609424 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.609409 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.609461 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.609441 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddd38197-2fcb-4086-a376-b5ef4628be3c-original-pull-secret\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:59.609492 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.609441 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.617921 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.617905 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad2cc585-2e96-43fe-b778-f19bb57ac50d-cni-binary-copy\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.617990 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.617925 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.622571 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.622548 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnjmq\" (UniqueName: \"kubernetes.io/projected/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-kube-api-access-dnjmq\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.622673 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.622639 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s778k\" (UniqueName: \"kubernetes.io/projected/d282c492-0fef-42cf-922e-86b1355221f2-kube-api-access-s778k\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.622729 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.622670 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr56b\" (UniqueName: \"kubernetes.io/projected/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-kube-api-access-hr56b\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:59.622729 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.622717 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf6dc\" (UniqueName: \"kubernetes.io/projected/b94ca77d-354d-4404-99a9-dd6dada01c24-kube-api-access-sf6dc\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:59.622949 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.622935 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v85z\" (UniqueName: \"kubernetes.io/projected/09c6a70b-bc11-4b06-a052-ca93a697355b-kube-api-access-9v85z\") pod \"network-check-target-z99g5\" (UID: \"09c6a70b-bc11-4b06-a052-ca93a697355b\") " pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:45:59.623476 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.623461 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgt87\" (UniqueName: \"kubernetes.io/projected/53eaa748-5fc9-4d33-8fd4-946c97ec0074-kube-api-access-dgt87\") pod \"network-metrics-daemon-m87zm\" (UID: \"53eaa748-5fc9-4d33-8fd4-946c97ec0074\") " pod="openshift-multus/network-metrics-daemon-m87zm" Nov 04 04:45:59.623706 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.623691 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nbph\" (UniqueName: \"kubernetes.io/projected/ad2cc585-2e96-43fe-b778-f19bb57ac50d-kube-api-access-4nbph\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.624783 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.624767 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfb82\" (UniqueName: \"kubernetes.io/projected/b2224b84-c10c-46e8-b7ba-844bc5802f34-kube-api-access-bfb82\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:59.624837 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.624773 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tstz6\" (UniqueName: \"kubernetes.io/projected/4affa996-4479-46e2-bde8-dba1989da65d-kube-api-access-tstz6\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.624948 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.624933 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knj8p\" (UniqueName: \"kubernetes.io/projected/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-kube-api-access-knj8p\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.670323 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670297 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-slash\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.670323 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670323 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-node-log\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.670483 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670337 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-host\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:59.670483 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670354 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b94ca77d-354d-4404-99a9-dd6dada01c24-hosts-file\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:59.670483 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670371 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-slash\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.670483 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670379 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-system-cni-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.670483 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670383 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-node-log\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.670483 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670403 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58c2628d-e8f4-4dd4-987e-c7c4f3847fac-host\") pod \"node-ca-n278n\" (UID: \"58c2628d-e8f4-4dd4-987e-c7c4f3847fac\") " pod="openshift-image-registry/node-ca-n278n" Nov 04 04:45:59.670483 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670413 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-netns\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.670483 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670432 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-system-cni-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.670483 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670445 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b94ca77d-354d-4404-99a9-dd6dada01c24-hosts-file\") pod \"node-resolver-b4n8j\" (UID: \"b94ca77d-354d-4404-99a9-dd6dada01c24\") " pod="openshift-dns/node-resolver-b4n8j" Nov 04 04:45:59.670483 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670451 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ddd38197-2fcb-4086-a376-b5ef4628be3c-dbus\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670493 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ddd38197-2fcb-4086-a376-b5ef4628be3c-dbus\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670498 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-registration-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670454 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-netns\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670530 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670558 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ddd38197-2fcb-4086-a376-b5ef4628be3c-kubelet-config\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670583 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-run-netns\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670601 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670610 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-etc-kubernetes\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670616 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-registration-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670634 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-run-netns\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670641 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-etc-kubernetes\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670634 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-etc-selinux\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670620 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ddd38197-2fcb-4086-a376-b5ef4628be3c-kubelet-config\") pod \"global-pull-secret-syncer-stlg9\" (UID: \"ddd38197-2fcb-4086-a376-b5ef4628be3c\") " pod="kube-system/global-pull-secret-syncer-stlg9" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670667 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-sys-fs\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670680 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-etc-selinux\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670682 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-systemd-units\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670706 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-system-cni-dir\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670721 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cnibin\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670735 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysconfig\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670750 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysctl-d\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670753 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-system-cni-dir\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670763 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-socket-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670721 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-sys-fs\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670776 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-systemd-units\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670779 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-sys\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670805 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-var-lib-kubelet\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670825 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-sys\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670831 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-socket-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670824 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysconfig\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670827 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-lib-modules\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670882 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysctl-d\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670893 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2224b84-c10c-46e8-b7ba-844bc5802f34-host-slash\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670916 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-cni-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670917 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2224b84-c10c-46e8-b7ba-844bc5802f34-host-slash\") pod \"iptables-alerter-7n7vn\" (UID: \"b2224b84-c10c-46e8-b7ba-844bc5802f34\") " pod="openshift-network-operator/iptables-alerter-7n7vn" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670927 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-lib-modules\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670930 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-kubelet\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670892 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-cnibin\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670948 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-conf-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670962 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-cni-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670927 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-var-lib-kubelet\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670968 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysctl-conf\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670987 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-kubelet\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670996 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-conf-dir\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.670996 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-cni-bin\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671025 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-socket-dir-parent\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671022 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-cni-bin\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.671017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671045 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671062 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-device-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671078 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-kubelet\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671081 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671076 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-multus-socket-dir-parent\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671080 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-sysctl-conf\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671113 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-kubelet\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671113 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d282c492-0fef-42cf-922e-86b1355221f2-device-dir\") pod \"aws-ebs-csi-driver-node-phz2v\" (UID: \"d282c492-0fef-42cf-922e-86b1355221f2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2v" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671143 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-var-lib-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671169 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-systemd\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671187 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-k8s-cni-cncf-io\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671212 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-k8s-cni-cncf-io\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671169 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-var-lib-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671235 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-systemd\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671268 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671294 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-kubernetes\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671303 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671318 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-cni-bin\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671331 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-kubernetes\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671342 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-ovn\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671353 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-cni-bin\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671375 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671400 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/456c901b-f6fa-4de0-85aa-59d4fc1ec5c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvpj9\" (UID: \"456c901b-f6fa-4de0-85aa-59d4fc1ec5c0\") " pod="openshift-multus/multus-additional-cni-plugins-mvpj9" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671403 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-cni-multus\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671429 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-etc-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671438 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-etc-openvswitch\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671378 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-ovn\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671467 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-var-lib-cni-multus\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671454 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-cni-netd\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671497 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-cnibin\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671510 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-hostroot\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671523 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-systemd\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671535 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-cni-netd\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671537 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-run-ovn-kubernetes\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671555 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-host-run-ovn-kubernetes\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671566 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-cnibin\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671575 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-run\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671596 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-multus-certs\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671611 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-log-socket\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671576 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-run-systemd\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671624 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-modprobe-d\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671641 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd6b4c99-03f6-42fb-9a97-ff51e2efb90c-log-socket\") pod \"ovnkube-node-j7pdj\" (UID: \"bd6b4c99-03f6-42fb-9a97-ff51e2efb90c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671642 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-host\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671640 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-run\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671644 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-host-run-multus-certs\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671669 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-host\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671549 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad2cc585-2e96-43fe-b778-f19bb57ac50d-hostroot\") pod \"multus-lfwvp\" (UID: \"ad2cc585-2e96-43fe-b778-f19bb57ac50d\") " pod="openshift-multus/multus-lfwvp" Nov 04 04:45:59.672693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:45:59.671705 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4affa996-4479-46e2-bde8-dba1989da65d-etc-modprobe-d\") pod \"tuned-jkgz4\" (UID: \"4affa996-4479-46e2-bde8-dba1989da65d\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgz4" Nov 04 04:46:00.822164 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:00.822118 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:01.012172 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:01.012135 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:01.154058 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:01.153970 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:01.555553 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:01.555530 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:46:01.973662 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:01.973628 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:02.170963 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:02.170927 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:02.634123 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:02.634090 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:46:02.634308 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:02.634295 4374 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 04:46:02.761688 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:02.761658 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:02.903659 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:02.903589 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:03.065136 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:03.065110 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:03.213650 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:03.213576 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:03.363043 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:03.363017 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:03.915552 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:03.915520 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7pdj" Nov 04 04:46:04.070018 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:04.069982 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:04.241678 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:04.241642 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:04.383733 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:04.383707 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:05.430999 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:05.430969 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:05.571314 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:05.571229 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:07.069256 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:07.069221 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:07.211439 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:07.211406 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:09.131057 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.131031 4374 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-242.us-west-2.compute.internal" event="NodeReady" Nov 04 04:46:09.276490 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.276450 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:09.291157 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.291129 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gt8km"] Nov 04 04:46:09.293952 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.293929 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f5mnr"] Nov 04 04:46:09.294085 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.294042 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.296455 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.296434 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xrmjj"] Nov 04 04:46:09.296552 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.296526 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.298918 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.298901 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xrmjj" Nov 04 04:46:09.317697 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.317678 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f5mnr"] Nov 04 04:46:09.321527 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.321506 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gt8km"] Nov 04 04:46:09.326003 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.325978 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1998a8c5-6ee0-400c-8c94-87e650ec8fee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.326093 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.326022 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h8fd\" (UniqueName: \"kubernetes.io/projected/1998a8c5-6ee0-400c-8c94-87e650ec8fee-kube-api-access-6h8fd\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.326129 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.326099 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqqrc\" (UniqueName: \"kubernetes.io/projected/a81a014c-ab71-40a3-ad7c-98002411005d-kube-api-access-kqqrc\") pod \"ingress-canary-xrmjj\" (UID: \"a81a014c-ab71-40a3-ad7c-98002411005d\") " pod="openshift-ingress-canary/ingress-canary-xrmjj" Nov 04 04:46:09.326162 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.326134 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbjfl\" (UniqueName: \"kubernetes.io/projected/dfe19b9e-9eb7-49b5-addb-74ec0df10e0a-kube-api-access-jbjfl\") pod \"dns-default-f5mnr\" (UID: \"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a\") " pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.326195 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.326164 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b9e-9eb7-49b5-addb-74ec0df10e0a-tmp-dir\") pod \"dns-default-f5mnr\" (UID: \"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a\") " pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.326315 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.326291 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1998a8c5-6ee0-400c-8c94-87e650ec8fee-data-volume\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.326427 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.326369 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dfe19b9e-9eb7-49b5-addb-74ec0df10e0a-metrics-tls\") pod \"dns-default-f5mnr\" (UID: \"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a\") " pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.326427 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.326396 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1998a8c5-6ee0-400c-8c94-87e650ec8fee-crio-socket\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.326509 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.326438 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1998a8c5-6ee0-400c-8c94-87e650ec8fee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.326509 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.326466 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dfe19b9e-9eb7-49b5-addb-74ec0df10e0a-config-volume\") pod \"dns-default-f5mnr\" (UID: \"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a\") " pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.326580 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.326528 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a81a014c-ab71-40a3-ad7c-98002411005d-cert\") pod \"ingress-canary-xrmjj\" (UID: \"a81a014c-ab71-40a3-ad7c-98002411005d\") " pod="openshift-ingress-canary/ingress-canary-xrmjj" Nov 04 04:46:09.336295 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.336267 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xrmjj"] Nov 04 04:46:09.364410 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.364383 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 04:46:09.364410 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.364394 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 04:46:09.364570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.364436 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 04:46:09.364570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.364436 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 04:46:09.364570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.364486 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-frtvg\"" Nov 04 04:46:09.364570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.364496 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 04:46:09.364570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.364568 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 04:46:09.365394 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.365374 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 04:46:09.365501 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.365489 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 04:46:09.366976 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.366964 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-4k9ql\"" Nov 04 04:46:09.367021 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.366983 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 04:46:09.367021 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.367000 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jn85r\"" Nov 04 04:46:09.424960 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.424878 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:09.426937 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.426915 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1998a8c5-6ee0-400c-8c94-87e650ec8fee-crio-socket\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.427032 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.426942 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1998a8c5-6ee0-400c-8c94-87e650ec8fee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.427032 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.426962 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dfe19b9e-9eb7-49b5-addb-74ec0df10e0a-config-volume\") pod \"dns-default-f5mnr\" (UID: \"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a\") " pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.427032 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.426987 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a81a014c-ab71-40a3-ad7c-98002411005d-cert\") pod \"ingress-canary-xrmjj\" (UID: \"a81a014c-ab71-40a3-ad7c-98002411005d\") " pod="openshift-ingress-canary/ingress-canary-xrmjj" Nov 04 04:46:09.427131 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427031 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1998a8c5-6ee0-400c-8c94-87e650ec8fee-crio-socket\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.427131 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427096 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1998a8c5-6ee0-400c-8c94-87e650ec8fee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.427186 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427133 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h8fd\" (UniqueName: \"kubernetes.io/projected/1998a8c5-6ee0-400c-8c94-87e650ec8fee-kube-api-access-6h8fd\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.427186 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427160 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqqrc\" (UniqueName: \"kubernetes.io/projected/a81a014c-ab71-40a3-ad7c-98002411005d-kube-api-access-kqqrc\") pod \"ingress-canary-xrmjj\" (UID: \"a81a014c-ab71-40a3-ad7c-98002411005d\") " pod="openshift-ingress-canary/ingress-canary-xrmjj" Nov 04 04:46:09.427260 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427193 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbjfl\" (UniqueName: \"kubernetes.io/projected/dfe19b9e-9eb7-49b5-addb-74ec0df10e0a-kube-api-access-jbjfl\") pod \"dns-default-f5mnr\" (UID: \"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a\") " pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.427260 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427225 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b9e-9eb7-49b5-addb-74ec0df10e0a-tmp-dir\") pod \"dns-default-f5mnr\" (UID: \"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a\") " pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.427331 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427264 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1998a8c5-6ee0-400c-8c94-87e650ec8fee-data-volume\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.427331 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427310 4374 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 04:46:09.427507 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427319 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dfe19b9e-9eb7-49b5-addb-74ec0df10e0a-metrics-tls\") pod \"dns-default-f5mnr\" (UID: \"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a\") " pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.427627 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427594 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dfe19b9e-9eb7-49b5-addb-74ec0df10e0a-config-volume\") pod \"dns-default-f5mnr\" (UID: \"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a\") " pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.427712 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427690 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1998a8c5-6ee0-400c-8c94-87e650ec8fee-data-volume\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.427813 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427728 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfe19b9e-9eb7-49b5-addb-74ec0df10e0a-tmp-dir\") pod \"dns-default-f5mnr\" (UID: \"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a\") " pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.427813 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.427740 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1998a8c5-6ee0-400c-8c94-87e650ec8fee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.429516 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.429496 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dfe19b9e-9eb7-49b5-addb-74ec0df10e0a-metrics-tls\") pod \"dns-default-f5mnr\" (UID: \"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a\") " pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.429516 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.429513 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1998a8c5-6ee0-400c-8c94-87e650ec8fee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.429748 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.429731 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a81a014c-ab71-40a3-ad7c-98002411005d-cert\") pod \"ingress-canary-xrmjj\" (UID: \"a81a014c-ab71-40a3-ad7c-98002411005d\") " pod="openshift-ingress-canary/ingress-canary-xrmjj" Nov 04 04:46:09.441522 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.441503 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:46:09.484019 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.483995 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbjfl\" (UniqueName: \"kubernetes.io/projected/dfe19b9e-9eb7-49b5-addb-74ec0df10e0a-kube-api-access-jbjfl\") pod \"dns-default-f5mnr\" (UID: \"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a\") " pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.484109 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.484041 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqqrc\" (UniqueName: \"kubernetes.io/projected/a81a014c-ab71-40a3-ad7c-98002411005d-kube-api-access-kqqrc\") pod \"ingress-canary-xrmjj\" (UID: \"a81a014c-ab71-40a3-ad7c-98002411005d\") " pod="openshift-ingress-canary/ingress-canary-xrmjj" Nov 04 04:46:09.484601 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.484585 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h8fd\" (UniqueName: \"kubernetes.io/projected/1998a8c5-6ee0-400c-8c94-87e650ec8fee-kube-api-access-6h8fd\") pod \"insights-runtime-extractor-gt8km\" (UID: \"1998a8c5-6ee0-400c-8c94-87e650ec8fee\") " pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.610267 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.610246 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gt8km" Nov 04 04:46:09.614021 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.614001 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:09.616133 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:09.616105 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xrmjj" Nov 04 04:46:10.055248 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.055210 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xrmjj"] Nov 04 04:46:10.059372 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.059214 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f5mnr"] Nov 04 04:46:10.059372 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.059276 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gt8km"] Nov 04 04:46:10.060237 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:46:10.060211 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81a014c_ab71_40a3_ad7c_98002411005d.slice/crio-a9372530d0992d63293dd1eb5db3e9aae6d4ea423ff40f1f5616bbee28bc016e WatchSource:0}: Error finding container a9372530d0992d63293dd1eb5db3e9aae6d4ea423ff40f1f5616bbee28bc016e: Status 404 returned error can't find the container with id a9372530d0992d63293dd1eb5db3e9aae6d4ea423ff40f1f5616bbee28bc016e Nov 04 04:46:10.062638 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.062615 4374 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 04:46:10.063233 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:46:10.063214 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1998a8c5_6ee0_400c_8c94_87e650ec8fee.slice/crio-407afffb9cd284fb383110c4b6b2045208067eb05ec2ebc73a3b7e0dbad55da2 WatchSource:0}: Error finding container 407afffb9cd284fb383110c4b6b2045208067eb05ec2ebc73a3b7e0dbad55da2: Status 404 returned error can't find the container with id 407afffb9cd284fb383110c4b6b2045208067eb05ec2ebc73a3b7e0dbad55da2 Nov 04 04:46:10.063903 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:46:10.063785 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfe19b9e_9eb7_49b5_addb_74ec0df10e0a.slice/crio-2073700b9dfb7fbf8104941cc92fd6624f6187cb4bbeb0b9eea5cf194d6a9631 WatchSource:0}: Error finding container 2073700b9dfb7fbf8104941cc92fd6624f6187cb4bbeb0b9eea5cf194d6a9631: Status 404 returned error can't find the container with id 2073700b9dfb7fbf8104941cc92fd6624f6187cb4bbeb0b9eea5cf194d6a9631 Nov 04 04:46:10.279952 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.279914 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:10.421163 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.421087 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:10.467137 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.467108 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f5mnr" event={"ID":"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a","Type":"ContainerStarted","Data":"2073700b9dfb7fbf8104941cc92fd6624f6187cb4bbeb0b9eea5cf194d6a9631"} Nov 04 04:46:10.468614 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.468590 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gt8km" event={"ID":"1998a8c5-6ee0-400c-8c94-87e650ec8fee","Type":"ContainerStarted","Data":"71e044ab4d876129f850284248865b5881bb306c0b2e77f13f78fee5c9a5f0b2"} Nov 04 04:46:10.468701 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.468641 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gt8km" event={"ID":"1998a8c5-6ee0-400c-8c94-87e650ec8fee","Type":"ContainerStarted","Data":"407afffb9cd284fb383110c4b6b2045208067eb05ec2ebc73a3b7e0dbad55da2"} Nov 04 04:46:10.469669 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.469649 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xrmjj" event={"ID":"a81a014c-ab71-40a3-ad7c-98002411005d","Type":"ContainerStarted","Data":"a9372530d0992d63293dd1eb5db3e9aae6d4ea423ff40f1f5616bbee28bc016e"} Nov 04 04:46:10.570362 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.570331 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:10.676486 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.676431 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8zd4x" Nov 04 04:46:10.714212 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:10.714188 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:12.476697 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:12.476666 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gt8km" event={"ID":"1998a8c5-6ee0-400c-8c94-87e650ec8fee","Type":"ContainerStarted","Data":"221d46b38ea030e86b4b6030a5fcce27a5f51c08fc08cab1317a29a2db541bae"} Nov 04 04:46:13.481584 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:13.481551 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f5mnr" event={"ID":"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a","Type":"ContainerStarted","Data":"229c5d0c3b6b83367bf2f813b871e952864e6a5265d8fff586b5dacb1a413e7d"} Nov 04 04:46:13.481970 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:13.481601 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f5mnr" event={"ID":"dfe19b9e-9eb7-49b5-addb-74ec0df10e0a","Type":"ContainerStarted","Data":"41ada0abf539ece51ea5173395e9a499e119e63f02757be3dc9ea6e072debf58"} Nov 04 04:46:13.481970 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:13.481675 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:16.494560 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:16.494528 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gt8km" event={"ID":"1998a8c5-6ee0-400c-8c94-87e650ec8fee","Type":"ContainerStarted","Data":"8d59b85637010a8369a84c83d58863044831fb2ecd25cc657973beb0fbd1ef55"} Nov 04 04:46:16.605697 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:16.605640 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gt8km" podStartSLOduration=1.903615559 podStartE2EDuration="7.605620715s" totalImagesPullingTime="5.702005156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:46:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:46:16.605162593 +0000 UTC m=+18.316853754" watchObservedRunningTime="2026-11-04 04:46:16.605620715 +0000 UTC m=+18.317311870" Nov 04 04:46:16.605921 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:16.605894 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-f5mnr" podStartSLOduration=4.702950566 podStartE2EDuration="7.605887783s" totalImagesPullingTime="2.902937217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:46:13.591218095 +0000 UTC m=+15.302909238" watchObservedRunningTime="2026-11-04 04:46:16.605887783 +0000 UTC m=+18.317578921" Nov 04 04:46:17.194368 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:17.194292 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:17.499523 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:17.499490 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xrmjj" event={"ID":"a81a014c-ab71-40a3-ad7c-98002411005d","Type":"ContainerStarted","Data":"323919f8a566e2649c78104dc2be7cbeac604abca2737373d73bdce5e2764dff"} Nov 04 04:46:17.608230 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:17.608184 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xrmjj" podStartSLOduration=1.851147892 podStartE2EDuration="8.60816847s" totalImagesPullingTime="6.757020578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:46:17.607859651 +0000 UTC m=+19.319550790" watchObservedRunningTime="2026-11-04 04:46:17.60816847 +0000 UTC m=+19.319859608" Nov 04 04:46:17.747234 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:17.747200 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:17.894093 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:17.894023 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:18.036542 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.036514 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:18.319917 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.319889 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:18.466331 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.466292 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:18.541075 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.541047 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6"] Nov 04 04:46:18.544121 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.544101 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.591199 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.591139 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bede814c-5c34-409e-a795-1cc50f629257-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-4hdx6\" (UID: \"bede814c-5c34-409e-a795-1cc50f629257\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.591199 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.591186 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bede814c-5c34-409e-a795-1cc50f629257-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-4hdx6\" (UID: \"bede814c-5c34-409e-a795-1cc50f629257\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.591349 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.591283 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bede814c-5c34-409e-a795-1cc50f629257-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-4hdx6\" (UID: \"bede814c-5c34-409e-a795-1cc50f629257\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.591349 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.591326 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s792b\" (UniqueName: \"kubernetes.io/projected/bede814c-5c34-409e-a795-1cc50f629257-kube-api-access-s792b\") pod \"openshift-state-metrics-7bd7489957-4hdx6\" (UID: \"bede814c-5c34-409e-a795-1cc50f629257\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.593924 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.593905 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 04:46:18.612201 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.612175 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:18.640208 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.640185 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Nov 04 04:46:18.640312 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.640221 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 04:46:18.640483 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.640463 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Nov 04 04:46:18.640566 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.640501 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9m7z4\"" Nov 04 04:46:18.640566 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.640534 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 04:46:18.641244 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.641225 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6"] Nov 04 04:46:18.687524 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.687494 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf"] Nov 04 04:46:18.690461 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.690445 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kvd59"] Nov 04 04:46:18.690610 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.690593 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.691656 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.691634 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bede814c-5c34-409e-a795-1cc50f629257-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-4hdx6\" (UID: \"bede814c-5c34-409e-a795-1cc50f629257\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.691724 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.691685 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s792b\" (UniqueName: \"kubernetes.io/projected/bede814c-5c34-409e-a795-1cc50f629257-kube-api-access-s792b\") pod \"openshift-state-metrics-7bd7489957-4hdx6\" (UID: \"bede814c-5c34-409e-a795-1cc50f629257\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.691724 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.691716 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bede814c-5c34-409e-a795-1cc50f629257-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-4hdx6\" (UID: \"bede814c-5c34-409e-a795-1cc50f629257\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.691803 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.691744 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bede814c-5c34-409e-a795-1cc50f629257-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-4hdx6\" (UID: \"bede814c-5c34-409e-a795-1cc50f629257\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.692401 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.692382 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bede814c-5c34-409e-a795-1cc50f629257-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-4hdx6\" (UID: \"bede814c-5c34-409e-a795-1cc50f629257\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.693676 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.693655 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.694186 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.694164 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bede814c-5c34-409e-a795-1cc50f629257-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-4hdx6\" (UID: \"bede814c-5c34-409e-a795-1cc50f629257\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.694309 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.694291 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bede814c-5c34-409e-a795-1cc50f629257-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-4hdx6\" (UID: \"bede814c-5c34-409e-a795-1cc50f629257\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.748514 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.748486 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s792b\" (UniqueName: \"kubernetes.io/projected/bede814c-5c34-409e-a795-1cc50f629257-kube-api-access-s792b\") pod \"openshift-state-metrics-7bd7489957-4hdx6\" (UID: \"bede814c-5c34-409e-a795-1cc50f629257\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.756962 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.755785 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf"] Nov 04 04:46:18.757355 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.757321 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:18.792799 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.792764 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.792799 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.792801 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-tls\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.793023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.792820 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-wtmp\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.793023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.792838 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5qbk\" (UniqueName: \"kubernetes.io/projected/7ccb6ee6-7eff-44c5-a207-a5ab97892976-kube-api-access-g5qbk\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.793023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.792920 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.793149 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.793026 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-textfile\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.793149 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.793052 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.793256 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.793165 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-accelerators-collector-config\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.793256 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.793207 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84l8s\" (UniqueName: \"kubernetes.io/projected/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-kube-api-access-84l8s\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.793256 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.793243 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ccb6ee6-7eff-44c5-a207-a5ab97892976-sys\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.793401 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.793331 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.793401 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.793363 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.793401 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.793396 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ccb6ee6-7eff-44c5-a207-a5ab97892976-root\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.793508 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.793471 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ccb6ee6-7eff-44c5-a207-a5ab97892976-metrics-client-ca\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.793508 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.793494 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.813608 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.813577 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Nov 04 04:46:18.813608 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.813591 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 04:46:18.813784 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.813619 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vb2v5\"" Nov 04 04:46:18.814032 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.814013 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 04:46:18.814148 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.814131 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Nov 04 04:46:18.814202 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.814150 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Nov 04 04:46:18.814202 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.814153 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mjbxn\"" Nov 04 04:46:18.814202 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.814163 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 04:46:18.858886 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.858816 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" Nov 04 04:46:18.894182 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894148 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ccb6ee6-7eff-44c5-a207-a5ab97892976-sys\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.894346 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894192 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.894346 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894220 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.894346 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894245 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ccb6ee6-7eff-44c5-a207-a5ab97892976-root\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.894346 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894247 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ccb6ee6-7eff-44c5-a207-a5ab97892976-sys\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.894346 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894263 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ccb6ee6-7eff-44c5-a207-a5ab97892976-metrics-client-ca\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.894346 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894307 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.894346 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894341 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.894714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894356 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ccb6ee6-7eff-44c5-a207-a5ab97892976-root\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.894714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894371 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-tls\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.894714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894416 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-wtmp\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.894714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894446 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5qbk\" (UniqueName: \"kubernetes.io/projected/7ccb6ee6-7eff-44c5-a207-a5ab97892976-kube-api-access-g5qbk\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.894714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894473 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.894714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894498 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-textfile\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.894714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894518 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.894714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894563 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-accelerators-collector-config\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.894714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894591 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84l8s\" (UniqueName: \"kubernetes.io/projected/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-kube-api-access-84l8s\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.894714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.894634 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-wtmp\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.895212 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.895161 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.895448 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.895424 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ccb6ee6-7eff-44c5-a207-a5ab97892976-metrics-client-ca\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.895520 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.895451 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-accelerators-collector-config\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.895520 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.895464 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-textfile\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.896037 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.896012 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.896297 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.896271 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.898137 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.898104 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.898649 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.898629 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.898738 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.898661 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:18.899389 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.899371 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ccb6ee6-7eff-44c5-a207-a5ab97892976-node-exporter-tls\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.905608 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.905588 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:18.956854 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.956829 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5qbk\" (UniqueName: \"kubernetes.io/projected/7ccb6ee6-7eff-44c5-a207-a5ab97892976-kube-api-access-g5qbk\") pod \"node-exporter-kvd59\" (UID: \"7ccb6ee6-7eff-44c5-a207-a5ab97892976\") " pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:18.957051 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:18.957034 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84l8s\" (UniqueName: \"kubernetes.io/projected/853f9b0c-b2ed-4f8a-9d27-1066ac8063a3-kube-api-access-84l8s\") pod \"kube-state-metrics-5dc8d7d89b-2nggf\" (UID: \"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:19.012163 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.012140 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" Nov 04 04:46:19.018298 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.018280 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kvd59" Nov 04 04:46:19.028179 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:46:19.028154 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ccb6ee6_7eff_44c5_a207_a5ab97892976.slice/crio-19fd993e814447a2403a7b20542979bb05cacbd7eb8029412a03584be81f2a9d WatchSource:0}: Error finding container 19fd993e814447a2403a7b20542979bb05cacbd7eb8029412a03584be81f2a9d: Status 404 returned error can't find the container with id 19fd993e814447a2403a7b20542979bb05cacbd7eb8029412a03584be81f2a9d Nov 04 04:46:19.138651 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.138581 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:19.273333 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.273304 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6"] Nov 04 04:46:19.276692 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:46:19.276663 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbede814c_5c34_409e_a795_1cc50f629257.slice/crio-7c4ef2c30389db626b802fbe23ba7b2f69d44dd5ed9e888f3de21a5f5f5fddd4 WatchSource:0}: Error finding container 7c4ef2c30389db626b802fbe23ba7b2f69d44dd5ed9e888f3de21a5f5f5fddd4: Status 404 returned error can't find the container with id 7c4ef2c30389db626b802fbe23ba7b2f69d44dd5ed9e888f3de21a5f5f5fddd4 Nov 04 04:46:19.374080 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.374053 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf"] Nov 04 04:46:19.377128 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:46:19.377101 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853f9b0c_b2ed_4f8a_9d27_1066ac8063a3.slice/crio-fe61b4c71c7c05181a9767755e96c119c750bcbb63df79695795b2fba78dc22f WatchSource:0}: Error finding container fe61b4c71c7c05181a9767755e96c119c750bcbb63df79695795b2fba78dc22f: Status 404 returned error can't find the container with id fe61b4c71c7c05181a9767755e96c119c750bcbb63df79695795b2fba78dc22f Nov 04 04:46:19.505680 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.505645 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" event={"ID":"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3","Type":"ContainerStarted","Data":"fe61b4c71c7c05181a9767755e96c119c750bcbb63df79695795b2fba78dc22f"} Nov 04 04:46:19.507534 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.507506 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" event={"ID":"bede814c-5c34-409e-a795-1cc50f629257","Type":"ContainerStarted","Data":"d9271c197b50fe434351395657b950b160cd3a129cbdd6219559d483867eb666"} Nov 04 04:46:19.507659 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.507559 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" event={"ID":"bede814c-5c34-409e-a795-1cc50f629257","Type":"ContainerStarted","Data":"764f696f37d5b8c016ad62580f35cfe08f00cba10471243af4c828582d8d489a"} Nov 04 04:46:19.507659 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.507579 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" event={"ID":"bede814c-5c34-409e-a795-1cc50f629257","Type":"ContainerStarted","Data":"7c4ef2c30389db626b802fbe23ba7b2f69d44dd5ed9e888f3de21a5f5f5fddd4"} Nov 04 04:46:19.508618 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.508597 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kvd59" event={"ID":"7ccb6ee6-7eff-44c5-a207-a5ab97892976","Type":"ContainerStarted","Data":"19fd993e814447a2403a7b20542979bb05cacbd7eb8029412a03584be81f2a9d"} Nov 04 04:46:19.685812 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.685779 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:19.828763 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.828691 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:19.972755 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:19.972726 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:20.114799 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.114728 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:20.847456 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.847422 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:20.939486 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.939449 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 04:46:20.942842 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.942821 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:20.994158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.994136 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 04 04:46:20.994158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.994158 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 04 04:46:20.997385 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.997360 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 04 04:46:20.998642 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.998222 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 04 04:46:20.998642 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.998366 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 04 04:46:20.998642 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.998439 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-l4gmp\"" Nov 04 04:46:20.998642 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.998557 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 04 04:46:20.998642 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.998581 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 04 04:46:20.998642 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:20.998609 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.006932 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.006974 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-config-volume\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.007000 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.007032 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.007058 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.007085 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-web-config\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.007110 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-config-out\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.007135 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.007184 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.007220 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.007251 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.007306 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftpdr\" (UniqueName: \"kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-kube-api-access-ftpdr\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.007578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.007329 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.091947 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.091855 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 04 04:46:21.092833 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.092802 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 04:46:21.108283 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.108211 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftpdr\" (UniqueName: \"kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-kube-api-access-ftpdr\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.108283 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.108242 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.108283 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.108261 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.108283 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.108280 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-config-volume\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.108571 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.108296 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.108571 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.108327 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.108571 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.108353 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.108571 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.108427 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-web-config\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.109489 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.108909 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-config-out\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.109489 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.108963 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.109489 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.109012 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.109489 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.109054 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.109489 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.109084 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.109814 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.109559 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.109949 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.109903 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.111330 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.111239 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.113083 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.113042 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-web-config\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.113673 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.113634 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.114676 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.114420 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-config-out\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.114676 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.114496 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.114676 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.114641 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.114906 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.114852 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-config-volume\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.115570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.115546 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.117283 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.117259 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.118144 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.118107 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.169709 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.169675 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftpdr\" (UniqueName: \"kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-kube-api-access-ftpdr\") pod \"alertmanager-main-0\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.268890 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.268836 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:46:21.517298 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.517262 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" event={"ID":"bede814c-5c34-409e-a795-1cc50f629257","Type":"ContainerStarted","Data":"32c99ec2dfa288fc23568c300a8c11c1d7da4c2dae10cb7a6e586d26df0ed464"} Nov 04 04:46:21.629107 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.628888 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-4hdx6" podStartSLOduration=1.579101715 podStartE2EDuration="3.6288579s" totalImagesPullingTime="2.049756185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:46:21.628671346 +0000 UTC m=+23.340362486" watchObservedRunningTime="2026-11-04 04:46:21.6288579 +0000 UTC m=+23.340549040" Nov 04 04:46:21.700152 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.700088 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 04:46:21.703469 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:46:21.703443 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda009301a_821d_4baa_b6c2_877574019d0c.slice/crio-71cf5d8844630c3cf0f3ccf6d0ebe8a35a364cf66d4afc0d0cf1dccd1b1d614e WatchSource:0}: Error finding container 71cf5d8844630c3cf0f3ccf6d0ebe8a35a364cf66d4afc0d0cf1dccd1b1d614e: Status 404 returned error can't find the container with id 71cf5d8844630c3cf0f3ccf6d0ebe8a35a364cf66d4afc0d0cf1dccd1b1d614e Nov 04 04:46:21.848423 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.848392 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:21.988617 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:21.988588 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:22.522681 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:22.522647 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" event={"ID":"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3","Type":"ContainerStarted","Data":"e6c5cca40523762a642c03b2cf3acd58c6f45c70907bac17d6f174810162da50"} Nov 04 04:46:22.522863 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:22.522712 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" event={"ID":"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3","Type":"ContainerStarted","Data":"ec935f8cc64c21b7a16e4ac3593a068991f59dd715fd789c371717fe0bc5d895"} Nov 04 04:46:22.522863 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:22.522746 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" event={"ID":"853f9b0c-b2ed-4f8a-9d27-1066ac8063a3","Type":"ContainerStarted","Data":"db31004aa96dd67596aa259c7fd3f4f11da9324c9ac60950d95ca70e9e40fbaf"} Nov 04 04:46:22.524266 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:22.524240 4374 generic.go:356] "Generic (PLEG): container finished" podID="7ccb6ee6-7eff-44c5-a207-a5ab97892976" containerID="8ff7f85da8a090c048b63e413e5445936d4c8b45cf5bd29638108235d64467b2" exitCode=0 Nov 04 04:46:22.524371 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:22.524321 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kvd59" event={"ID":"7ccb6ee6-7eff-44c5-a207-a5ab97892976","Type":"ContainerDied","Data":"8ff7f85da8a090c048b63e413e5445936d4c8b45cf5bd29638108235d64467b2"} Nov 04 04:46:22.525538 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:22.525513 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerStarted","Data":"71cf5d8844630c3cf0f3ccf6d0ebe8a35a364cf66d4afc0d0cf1dccd1b1d614e"} Nov 04 04:46:22.636110 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:22.636055 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-2nggf" podStartSLOduration=1.7308238500000002 podStartE2EDuration="4.63603701s" totalImagesPullingTime="2.90521316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:46:22.634671587 +0000 UTC m=+24.346362725" watchObservedRunningTime="2026-11-04 04:46:22.63603701 +0000 UTC m=+24.347728150" Nov 04 04:46:22.836509 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:22.836421 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:22.979491 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:22.979460 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:23.121359 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.121289 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:23.267279 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.267220 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:23.411546 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.411479 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:23.487513 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.487480 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f5mnr" Nov 04 04:46:23.523307 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.523273 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-bfb564b7d-p54wf"] Nov 04 04:46:23.529161 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.529133 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.532682 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.532652 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kvd59" event={"ID":"7ccb6ee6-7eff-44c5-a207-a5ab97892976","Type":"ContainerStarted","Data":"552fcee314a36c809b3f93d1598f43a5f6eb7bae4620476b6e7dfac86330181e"} Nov 04 04:46:23.532835 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.532705 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kvd59" event={"ID":"7ccb6ee6-7eff-44c5-a207-a5ab97892976","Type":"ContainerStarted","Data":"df1872254c2ee5a5390298c232737cd4bb8f3092cb5e997825f529c15a1fe7bc"} Nov 04 04:46:23.578040 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.578010 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Nov 04 04:46:23.578202 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.578166 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Nov 04 04:46:23.578259 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.578216 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Nov 04 04:46:23.578259 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.578229 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-firdn8jbgc8mb\"" Nov 04 04:46:23.581048 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.581029 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 04 04:46:23.582268 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.582245 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bfb564b7d-p54wf"] Nov 04 04:46:23.630560 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.630532 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0d1cc979-29c1-410d-b9a7-f339222dd71c-secret-metrics-server-client-certs\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.630970 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.630935 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0d1cc979-29c1-410d-b9a7-f339222dd71c-secret-metrics-server-tls\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.631083 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.631011 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0d1cc979-29c1-410d-b9a7-f339222dd71c-audit-log\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.631139 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.631115 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d1cc979-29c1-410d-b9a7-f339222dd71c-client-ca-bundle\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.631199 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.631151 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sqcb\" (UniqueName: \"kubernetes.io/projected/0d1cc979-29c1-410d-b9a7-f339222dd71c-kube-api-access-8sqcb\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.631383 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.631367 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0d1cc979-29c1-410d-b9a7-f339222dd71c-metrics-server-audit-profiles\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.631423 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.631398 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d1cc979-29c1-410d-b9a7-f339222dd71c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.678724 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.678666 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pgjf4\"" Nov 04 04:46:23.731802 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.731777 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0d1cc979-29c1-410d-b9a7-f339222dd71c-metrics-server-audit-profiles\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.731926 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.731815 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d1cc979-29c1-410d-b9a7-f339222dd71c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.731926 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.731900 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0d1cc979-29c1-410d-b9a7-f339222dd71c-secret-metrics-server-client-certs\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.732048 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.731942 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0d1cc979-29c1-410d-b9a7-f339222dd71c-secret-metrics-server-tls\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.732048 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.731972 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0d1cc979-29c1-410d-b9a7-f339222dd71c-audit-log\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.732048 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.732005 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d1cc979-29c1-410d-b9a7-f339222dd71c-client-ca-bundle\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.732048 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.732030 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sqcb\" (UniqueName: \"kubernetes.io/projected/0d1cc979-29c1-410d-b9a7-f339222dd71c-kube-api-access-8sqcb\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.732648 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.732588 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d1cc979-29c1-410d-b9a7-f339222dd71c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.732648 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.732603 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0d1cc979-29c1-410d-b9a7-f339222dd71c-audit-log\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.732944 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.732889 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0d1cc979-29c1-410d-b9a7-f339222dd71c-metrics-server-audit-profiles\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.734529 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.734508 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d1cc979-29c1-410d-b9a7-f339222dd71c-client-ca-bundle\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.734729 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.734709 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0d1cc979-29c1-410d-b9a7-f339222dd71c-secret-metrics-server-tls\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.735049 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.735020 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0d1cc979-29c1-410d-b9a7-f339222dd71c-secret-metrics-server-client-certs\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.789336 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.789313 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sqcb\" (UniqueName: \"kubernetes.io/projected/0d1cc979-29c1-410d-b9a7-f339222dd71c-kube-api-access-8sqcb\") pod \"metrics-server-bfb564b7d-p54wf\" (UID: \"0d1cc979-29c1-410d-b9a7-f339222dd71c\") " pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:23.819354 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.819309 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kvd59" podStartSLOduration=3.202329874 podStartE2EDuration="5.819294832s" totalImagesPullingTime="2.549410848s" totalInitContainerRuntime="67.55411ms" isStatefulPod=true podCreationTimestamp="2026-11-04 04:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:46:23.818224198 +0000 UTC m=+25.529915337" watchObservedRunningTime="2026-11-04 04:46:23.819294832 +0000 UTC m=+25.530985971" Nov 04 04:46:23.843552 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:23.843529 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:24.235660 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.235625 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-855956456c-pwkjc"] Nov 04 04:46:24.254389 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.254365 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.257535 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.257511 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bfb564b7d-p54wf"] Nov 04 04:46:24.285279 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:46:24.285249 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d1cc979_29c1_410d_b9a7_f339222dd71c.slice/crio-b56d5bb5bb7f5bcdaff42e0b33d7084245a279ef9076d33d01d0b8fa52197b35 WatchSource:0}: Error finding container b56d5bb5bb7f5bcdaff42e0b33d7084245a279ef9076d33d01d0b8fa52197b35: Status 404 returned error can't find the container with id b56d5bb5bb7f5bcdaff42e0b33d7084245a279ef9076d33d01d0b8fa52197b35 Nov 04 04:46:24.299025 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.298909 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-855956456c-pwkjc"] Nov 04 04:46:24.303462 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.303438 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Nov 04 04:46:24.303718 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.303466 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Nov 04 04:46:24.303718 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.303630 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Nov 04 04:46:24.303718 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.303644 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-7mm8r\"" Nov 04 04:46:24.303718 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.303633 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Nov 04 04:46:24.303990 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.303833 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Nov 04 04:46:24.335319 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.335296 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2d13baed-8251-49ef-bcd6-8818db2c6b22-secret-telemeter-client\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.335411 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.335324 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2d13baed-8251-49ef-bcd6-8818db2c6b22-federate-client-tls\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.335411 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.335347 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d13baed-8251-49ef-bcd6-8818db2c6b22-telemeter-trusted-ca-bundle\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.335411 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.335366 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d13baed-8251-49ef-bcd6-8818db2c6b22-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.335526 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.335479 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2d13baed-8251-49ef-bcd6-8818db2c6b22-telemeter-client-tls\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.335526 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.335504 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d13baed-8251-49ef-bcd6-8818db2c6b22-metrics-client-ca\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.335597 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.335525 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d13baed-8251-49ef-bcd6-8818db2c6b22-serving-certs-ca-bundle\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.335597 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.335544 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m86tq\" (UniqueName: \"kubernetes.io/projected/2d13baed-8251-49ef-bcd6-8818db2c6b22-kube-api-access-m86tq\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.350027 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.350009 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\"" Nov 04 04:46:24.436118 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.436058 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2d13baed-8251-49ef-bcd6-8818db2c6b22-secret-telemeter-client\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.436118 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.436085 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2d13baed-8251-49ef-bcd6-8818db2c6b22-federate-client-tls\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.436118 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.436108 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d13baed-8251-49ef-bcd6-8818db2c6b22-telemeter-trusted-ca-bundle\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.436350 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.436126 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d13baed-8251-49ef-bcd6-8818db2c6b22-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.436350 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.436157 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2d13baed-8251-49ef-bcd6-8818db2c6b22-telemeter-client-tls\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.436350 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.436175 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d13baed-8251-49ef-bcd6-8818db2c6b22-metrics-client-ca\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.436966 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.436304 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d13baed-8251-49ef-bcd6-8818db2c6b22-serving-certs-ca-bundle\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.437431 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.437070 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d13baed-8251-49ef-bcd6-8818db2c6b22-serving-certs-ca-bundle\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.437431 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.437086 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d13baed-8251-49ef-bcd6-8818db2c6b22-metrics-client-ca\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.437615 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.437589 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d13baed-8251-49ef-bcd6-8818db2c6b22-telemeter-trusted-ca-bundle\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.437741 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.437633 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m86tq\" (UniqueName: \"kubernetes.io/projected/2d13baed-8251-49ef-bcd6-8818db2c6b22-kube-api-access-m86tq\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.439011 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.438986 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2d13baed-8251-49ef-bcd6-8818db2c6b22-secret-telemeter-client\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.439011 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.439004 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d13baed-8251-49ef-bcd6-8818db2c6b22-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.439144 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.439042 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2d13baed-8251-49ef-bcd6-8818db2c6b22-federate-client-tls\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.439144 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.439062 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2d13baed-8251-49ef-bcd6-8818db2c6b22-telemeter-client-tls\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.494753 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.494729 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m86tq\" (UniqueName: \"kubernetes.io/projected/2d13baed-8251-49ef-bcd6-8818db2c6b22-kube-api-access-m86tq\") pod \"telemeter-client-855956456c-pwkjc\" (UID: \"2d13baed-8251-49ef-bcd6-8818db2c6b22\") " pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:24.538622 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.538594 4374 generic.go:356] "Generic (PLEG): container finished" podID="a009301a-821d-4baa-b6c2-877574019d0c" containerID="021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858" exitCode=0 Nov 04 04:46:24.538769 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.538681 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerDied","Data":"021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858"} Nov 04 04:46:24.539928 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.539903 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" event={"ID":"0d1cc979-29c1-410d-b9a7-f339222dd71c","Type":"ContainerStarted","Data":"b56d5bb5bb7f5bcdaff42e0b33d7084245a279ef9076d33d01d0b8fa52197b35"} Nov 04 04:46:24.617984 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:24.617952 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" Nov 04 04:46:25.031232 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.031202 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-855956456c-pwkjc"] Nov 04 04:46:25.034606 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:46:25.034581 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d13baed_8251_49ef_bcd6_8818db2c6b22.slice/crio-3f9447298af7058a62a398f9523205711190d4ad9dab3edb46b7f3ce06308454 WatchSource:0}: Error finding container 3f9447298af7058a62a398f9523205711190d4ad9dab3edb46b7f3ce06308454: Status 404 returned error can't find the container with id 3f9447298af7058a62a398f9523205711190d4ad9dab3edb46b7f3ce06308454 Nov 04 04:46:25.544198 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.544165 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" event={"ID":"2d13baed-8251-49ef-bcd6-8818db2c6b22","Type":"ContainerStarted","Data":"3f9447298af7058a62a398f9523205711190d4ad9dab3edb46b7f3ce06308454"} Nov 04 04:46:25.601655 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.601619 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d89979d7c-f6s98"] Nov 04 04:46:25.605289 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.605257 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.648574 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.648519 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-oauth-config\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.648722 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.648659 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-oauth-serving-cert\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.648782 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.648727 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-service-ca\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.648782 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.648760 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-config\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.648915 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.648818 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-trusted-ca-bundle\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.648972 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.648959 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-serving-cert\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.649149 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.649061 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-tmp\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.649260 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.649207 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzmjc\" (UniqueName: \"kubernetes.io/projected/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-kube-api-access-rzmjc\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.654003 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.653986 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Nov 04 04:46:25.654373 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.654351 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Nov 04 04:46:25.654503 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.654480 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Nov 04 04:46:25.654577 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.654489 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Nov 04 04:46:25.654634 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.654611 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7pqf8\"" Nov 04 04:46:25.654688 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.654647 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Nov 04 04:46:25.654688 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.654673 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Nov 04 04:46:25.654840 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.654712 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Nov 04 04:46:25.701378 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.701348 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Nov 04 04:46:25.702370 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.702348 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d89979d7c-f6s98"] Nov 04 04:46:25.749822 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.749794 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-oauth-serving-cert\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.749960 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.749840 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-service-ca\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.749960 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.749859 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-config\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.749960 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.749909 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-trusted-ca-bundle\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.749960 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.749952 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-serving-cert\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.750165 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.749972 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-tmp\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.750165 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.750007 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzmjc\" (UniqueName: \"kubernetes.io/projected/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-kube-api-access-rzmjc\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.750165 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.750054 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-oauth-config\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.750938 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.750630 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-config\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.750938 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.750671 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-oauth-serving-cert\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.750938 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.750691 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-tmp\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.750938 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.750832 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-trusted-ca-bundle\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.750938 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.750863 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-service-ca\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.752701 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.752675 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-serving-cert\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.752792 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.752705 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-oauth-config\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.807864 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.807810 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzmjc\" (UniqueName: \"kubernetes.io/projected/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-kube-api-access-rzmjc\") pod \"console-7d89979d7c-f6s98\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:25.833922 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:46:25.833889 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:46:25.922464 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:25.922407 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:26.293659 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:26.293613 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d89979d7c-f6s98"] Nov 04 04:46:26.323309 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:46:26.323284 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8edf2a1_5a5f_4ef6_bacf_b737a5733b5b.slice/crio-4f4f0e3ef17f9570bc4817702bf47e0a9418569168f303c9d3fc0e2aa54efd7d WatchSource:0}: Error finding container 4f4f0e3ef17f9570bc4817702bf47e0a9418569168f303c9d3fc0e2aa54efd7d: Status 404 returned error can't find the container with id 4f4f0e3ef17f9570bc4817702bf47e0a9418569168f303c9d3fc0e2aa54efd7d Nov 04 04:46:26.549741 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:26.549652 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d89979d7c-f6s98" event={"ID":"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b","Type":"ContainerStarted","Data":"4f4f0e3ef17f9570bc4817702bf47e0a9418569168f303c9d3fc0e2aa54efd7d"} Nov 04 04:46:27.194347 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:27.194315 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:27.335159 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:27.335128 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:27.515919 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:27.515885 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:27.565497 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:27.565411 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" event={"ID":"0d1cc979-29c1-410d-b9a7-f339222dd71c","Type":"ContainerStarted","Data":"5d1130de58f6c0b55c79b537da4079eec355b9a040c14814e8121b2d5eec8a14"} Nov 04 04:46:27.569983 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:27.569961 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" event={"ID":"2d13baed-8251-49ef-bcd6-8818db2c6b22","Type":"ContainerStarted","Data":"a2f3cf51f495efc7f98e3689a4964e392bd031ccb20d9df93a95851b1313d005"} Nov 04 04:46:27.676899 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:27.676158 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" podStartSLOduration=1.678270628 podStartE2EDuration="4.676141304s" totalImagesPullingTime="2.997870676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:46:27.675773145 +0000 UTC m=+29.387464296" watchObservedRunningTime="2026-11-04 04:46:27.676141304 +0000 UTC m=+29.387832444" Nov 04 04:46:27.683381 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:27.683352 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:27.844661 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:27.844634 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:28.576680 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:28.576644 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerStarted","Data":"ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c"} Nov 04 04:46:28.577088 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:28.576710 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerStarted","Data":"3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67"} Nov 04 04:46:28.577088 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:28.576747 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerStarted","Data":"c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a"} Nov 04 04:46:28.577088 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:28.576770 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerStarted","Data":"d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509"} Nov 04 04:46:28.577088 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:28.576792 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerStarted","Data":"e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15"} Nov 04 04:46:28.578701 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:28.578679 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" event={"ID":"2d13baed-8251-49ef-bcd6-8818db2c6b22","Type":"ContainerStarted","Data":"16af6d295450be8071845909f87d1030ee14426f7428d4c715d5f863691c7e7c"} Nov 04 04:46:28.578802 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:28.578718 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" event={"ID":"2d13baed-8251-49ef-bcd6-8818db2c6b22","Type":"ContainerStarted","Data":"3de9f576a50354e20c937610bcb070a28eb25e1bf693aa4c3a35c3212dd5b036"} Nov 04 04:46:28.700147 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:28.700089 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-855956456c-pwkjc" podStartSLOduration=2.3417735029999998 podStartE2EDuration="4.700075682s" totalImagesPullingTime="2.358302179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:46:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:46:28.699713698 +0000 UTC m=+30.411404831" watchObservedRunningTime="2026-11-04 04:46:28.700075682 +0000 UTC m=+30.411766856" Nov 04 04:46:29.444746 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:29.444714 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z99g5" Nov 04 04:46:30.591364 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:30.591330 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerStarted","Data":"891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7"} Nov 04 04:46:30.851281 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:30.851175 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.966484563 podStartE2EDuration="10.85115854s" totalImagesPullingTime="6.869725909s" totalInitContainerRuntime="14.948068ms" isStatefulPod=false podCreationTimestamp="2026-11-04 04:46:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:46:30.80442234 +0000 UTC m=+32.516113470" watchObservedRunningTime="2026-11-04 04:46:30.85115854 +0000 UTC m=+32.562849675" Nov 04 04:46:32.387846 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:32.387787 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:32.533197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:32.533173 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:32.601995 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:32.601958 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d89979d7c-f6s98" event={"ID":"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b","Type":"ContainerStarted","Data":"c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6"} Nov 04 04:46:32.750422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:32.750373 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7d89979d7c-f6s98" podStartSLOduration=1.590899152 podStartE2EDuration="7.750359541s" totalImagesPullingTime="6.159460389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:46:32.749506378 +0000 UTC m=+34.461197529" watchObservedRunningTime="2026-11-04 04:46:32.750359541 +0000 UTC m=+34.462050682" Nov 04 04:46:32.750812 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:32.750779 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:32.897645 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:32.897613 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:35.923552 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:35.923521 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:35.923940 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:35.923581 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:35.928380 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:35.928359 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:36.622813 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:36.619588 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:46:37.081262 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:37.081228 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:37.222240 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:37.222207 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:38.392798 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:38.392720 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:38.533340 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:38.533301 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:38.883619 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:38.883587 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:39.050068 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:39.050034 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:43.844320 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:43.844294 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:43.844840 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:43.844377 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:46:47.782769 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:47.782739 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d89979d7c-f6s98"] Nov 04 04:46:48.850263 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:48.850187 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:48.993195 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:48.993167 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:52.064276 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:52.064247 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:52.205418 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:52.205366 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:55.831631 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:46:55.831599 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:46:59.425912 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:59.425881 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:46:59.565699 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:46:59.565664 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:03.849147 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:03.849117 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:47:03.852762 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:03.852736 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-bfb564b7d-p54wf" Nov 04 04:47:07.062505 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:07.062473 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:07.208091 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:07.208055 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:12.802385 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:12.802343 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7d89979d7c-f6s98" podUID="d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" containerName="console" containerID="cri-o://c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6" gracePeriod=15 Nov 04 04:47:13.049863 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.049840 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:47:13.121278 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121199 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-oauth-serving-cert\") pod \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " Nov 04 04:47:13.121278 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121239 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-kube-api-access-rzmjc\" (UniqueName: \"kubernetes.io/projected/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-kube-api-access-rzmjc\") pod \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " Nov 04 04:47:13.121487 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121285 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-oauth-config\") pod \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " Nov 04 04:47:13.121487 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121313 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-service-ca\") pod \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " Nov 04 04:47:13.121487 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121347 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-trusted-ca-bundle\") pod \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " Nov 04 04:47:13.121487 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121375 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-config\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-config\") pod \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " Nov 04 04:47:13.121487 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121415 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-serving-cert\") pod \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " Nov 04 04:47:13.121487 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121441 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-tmp\") pod \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\" (UID: \"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b\") " Nov 04 04:47:13.121768 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121559 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-oauth-serving-cert" pod "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" (UID: "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:47:13.121825 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121779 4374 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-oauth-serving-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:13.121974 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121910 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-trusted-ca-bundle" pod "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" (UID: "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:47:13.121974 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121924 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-tmp" pod "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" (UID: "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:47:13.122204 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.121967 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-config" pod "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" (UID: "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:47:13.122204 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.122026 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-service-ca" pod "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" (UID: "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:47:13.123526 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.123502 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-oauth-config" pod "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" (UID: "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:47:13.123624 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.123572 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-kube-api-access-rzmjc" pod "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" (UID: "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b"). InnerVolumeSpecName "kube-api-access-rzmjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:47:13.123624 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.123589 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-serving-cert" pod "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" (UID: "d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:47:13.222771 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.222739 4374 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-oauth-config\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:13.222771 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.222764 4374 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-service-ca\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:13.222771 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.222773 4374 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-trusted-ca-bundle\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:13.223004 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.222782 4374 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-config\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:13.223004 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.222792 4374 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-console-serving-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:13.223004 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.222800 4374 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-tmp\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:13.223004 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.222810 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzmjc\" (UniqueName: \"kubernetes.io/projected/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b-kube-api-access-rzmjc\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:13.736113 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.736074 4374 generic.go:356] "Generic (PLEG): container finished" podID="d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" containerID="c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6" exitCode=0 Nov 04 04:47:13.736302 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.736120 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d89979d7c-f6s98" Nov 04 04:47:13.736302 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.736128 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d89979d7c-f6s98" event={"ID":"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b","Type":"ContainerDied","Data":"c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6"} Nov 04 04:47:13.736302 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.736176 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d89979d7c-f6s98" event={"ID":"d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b","Type":"ContainerDied","Data":"4f4f0e3ef17f9570bc4817702bf47e0a9418569168f303c9d3fc0e2aa54efd7d"} Nov 04 04:47:13.736302 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.736198 4374 scope.go:122] "RemoveContainer" containerID="c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6" Nov 04 04:47:13.744532 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.744514 4374 scope.go:122] "RemoveContainer" containerID="c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6" Nov 04 04:47:13.744801 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:47:13.744783 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6\": container with ID starting with c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6 not found: ID does not exist" containerID="c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6" Nov 04 04:47:13.744848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.744811 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6"} err="failed to get container status \"c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6\": rpc error: code = NotFound desc = could not find container \"c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6\": container with ID starting with c769f8bd58ab5e60ad7d08a36240978711980f6e87868b60196f0d8858119ce6 not found: ID does not exist" Nov 04 04:47:13.889005 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.888975 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d89979d7c-f6s98"] Nov 04 04:47:13.935504 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:13.935479 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d89979d7c-f6s98"] Nov 04 04:47:14.409649 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:14.409609 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" path="/var/lib/kubelet/pods/d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b/volumes" Nov 04 04:47:25.831121 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:47:25.831088 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:47:27.720652 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:27.720603 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:27.861532 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:27.861502 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:35.705200 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:35.705163 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 04:47:35.705740 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:35.705438 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="alertmanager" containerID="cri-o://e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15" gracePeriod=120 Nov 04 04:47:35.705740 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:35.705504 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="kube-rbac-proxy-metric" containerID="cri-o://ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c" gracePeriod=120 Nov 04 04:47:35.705740 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:35.705536 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="kube-rbac-proxy-web" containerID="cri-o://c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a" gracePeriod=120 Nov 04 04:47:35.705740 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:35.705565 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="config-reloader" containerID="cri-o://d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509" gracePeriod=120 Nov 04 04:47:35.705740 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:35.705600 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="kube-rbac-proxy" containerID="cri-o://3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67" gracePeriod=120 Nov 04 04:47:35.705740 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:35.705621 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="prom-label-proxy" containerID="cri-o://891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7" gracePeriod=120 Nov 04 04:47:36.820306 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:36.820277 4374 generic.go:356] "Generic (PLEG): container finished" podID="a009301a-821d-4baa-b6c2-877574019d0c" containerID="891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7" exitCode=0 Nov 04 04:47:36.820306 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:36.820308 4374 generic.go:356] "Generic (PLEG): container finished" podID="a009301a-821d-4baa-b6c2-877574019d0c" containerID="3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67" exitCode=0 Nov 04 04:47:36.820306 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:36.820315 4374 generic.go:356] "Generic (PLEG): container finished" podID="a009301a-821d-4baa-b6c2-877574019d0c" containerID="d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509" exitCode=0 Nov 04 04:47:36.820666 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:36.820322 4374 generic.go:356] "Generic (PLEG): container finished" podID="a009301a-821d-4baa-b6c2-877574019d0c" containerID="e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15" exitCode=0 Nov 04 04:47:36.820666 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:36.820346 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerDied","Data":"891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7"} Nov 04 04:47:36.820666 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:36.820405 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerDied","Data":"3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67"} Nov 04 04:47:36.820666 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:36.820431 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerDied","Data":"d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509"} Nov 04 04:47:36.820666 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:36.820455 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerDied","Data":"e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15"} Nov 04 04:47:36.952366 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:36.952345 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:37.089842 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.089771 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-tls-assets\" (UniqueName: \"kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-tls-assets\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.089842 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.089809 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-metrics-client-ca\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.089842 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.089828 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-cluster-tls-config\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.090131 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.089846 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-web-config\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-web-config\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.090131 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.089887 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-config-volume\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-config-volume\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.090131 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.089918 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-main-tls\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.090131 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.089978 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-config-out\" (UniqueName: \"kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-config-out\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.090131 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.090015 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-main-db\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.090131 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.090064 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.090434 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.090102 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-trusted-ca-bundle\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.090434 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.090177 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.090434 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.090210 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-metrics-client-ca" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:47:37.090434 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.090225 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-web\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.090434 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.090409 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-kube-api-access-ftpdr\" (UniqueName: \"kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-kube-api-access-ftpdr\") pod \"a009301a-821d-4baa-b6c2-877574019d0c\" (UID: \"a009301a-821d-4baa-b6c2-877574019d0c\") " Nov 04 04:47:37.090434 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.090416 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-main-db" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:47:37.090739 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.090695 4374 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-metrics-client-ca\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.090739 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.090716 4374 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-main-db\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.090863 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.090839 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-trusted-ca-bundle" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:47:37.092981 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.092954 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-tls-assets" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:47:37.094579 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.094541 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-web" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:47:37.094700 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.094656 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-kube-api-access-ftpdr" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "kube-api-access-ftpdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:47:37.094700 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.094673 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-config-volume" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:47:37.094939 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.094910 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-metric" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:47:37.094939 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.094921 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-config-out" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:47:37.095096 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.094947 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-main-tls" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:47:37.095096 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.094967 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:47:37.097201 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.097102 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-cluster-tls-config" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:47:37.102927 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.102907 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-web-config" pod "a009301a-821d-4baa-b6c2-877574019d0c" (UID: "a009301a-821d-4baa-b6c2-877574019d0c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:47:37.191187 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.191158 4374 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.191187 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.191185 4374 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a009301a-821d-4baa-b6c2-877574019d0c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.191343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.191195 4374 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.191343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.191205 4374 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.191343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.191215 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftpdr\" (UniqueName: \"kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-kube-api-access-ftpdr\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.191343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.191225 4374 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a009301a-821d-4baa-b6c2-877574019d0c-tls-assets\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.191343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.191233 4374 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-cluster-tls-config\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.191343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.191243 4374 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-web-config\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.191343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.191251 4374 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-config-volume\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.191343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.191260 4374 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a009301a-821d-4baa-b6c2-877574019d0c-secret-alertmanager-main-tls\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.191343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.191270 4374 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a009301a-821d-4baa-b6c2-877574019d0c-config-out\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:47:37.826589 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.826554 4374 generic.go:356] "Generic (PLEG): container finished" podID="a009301a-821d-4baa-b6c2-877574019d0c" containerID="ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c" exitCode=0 Nov 04 04:47:37.826589 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.826590 4374 generic.go:356] "Generic (PLEG): container finished" podID="a009301a-821d-4baa-b6c2-877574019d0c" containerID="c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a" exitCode=0 Nov 04 04:47:37.826961 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.826613 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerDied","Data":"ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c"} Nov 04 04:47:37.826961 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.826663 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:37.826961 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.826715 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerDied","Data":"c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a"} Nov 04 04:47:37.826961 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.826756 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a009301a-821d-4baa-b6c2-877574019d0c","Type":"ContainerDied","Data":"71cf5d8844630c3cf0f3ccf6d0ebe8a35a364cf66d4afc0d0cf1dccd1b1d614e"} Nov 04 04:47:37.826961 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.826785 4374 scope.go:122] "RemoveContainer" containerID="891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7" Nov 04 04:47:37.834117 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.834033 4374 scope.go:122] "RemoveContainer" containerID="ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c" Nov 04 04:47:37.842694 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.842676 4374 scope.go:122] "RemoveContainer" containerID="3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67" Nov 04 04:47:37.848997 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.848979 4374 scope.go:122] "RemoveContainer" containerID="c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a" Nov 04 04:47:37.855419 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.855402 4374 scope.go:122] "RemoveContainer" containerID="d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509" Nov 04 04:47:37.861675 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.861657 4374 scope.go:122] "RemoveContainer" containerID="e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15" Nov 04 04:47:37.868367 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.868349 4374 scope.go:122] "RemoveContainer" containerID="021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858" Nov 04 04:47:37.874468 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.874448 4374 scope.go:122] "RemoveContainer" containerID="891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7" Nov 04 04:47:37.874719 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:47:37.874701 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7\": container with ID starting with 891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7 not found: ID does not exist" containerID="891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7" Nov 04 04:47:37.874761 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.874728 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7"} err="failed to get container status \"891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7\": rpc error: code = NotFound desc = could not find container \"891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7\": container with ID starting with 891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7 not found: ID does not exist" Nov 04 04:47:37.874761 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.874745 4374 scope.go:122] "RemoveContainer" containerID="ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c" Nov 04 04:47:37.874987 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:47:37.874969 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c\": container with ID starting with ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c not found: ID does not exist" containerID="ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c" Nov 04 04:47:37.875035 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.874993 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c"} err="failed to get container status \"ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c\": rpc error: code = NotFound desc = could not find container \"ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c\": container with ID starting with ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c not found: ID does not exist" Nov 04 04:47:37.875035 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.875008 4374 scope.go:122] "RemoveContainer" containerID="3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67" Nov 04 04:47:37.875256 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:47:37.875239 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67\": container with ID starting with 3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67 not found: ID does not exist" containerID="3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67" Nov 04 04:47:37.875311 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.875264 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67"} err="failed to get container status \"3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67\": rpc error: code = NotFound desc = could not find container \"3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67\": container with ID starting with 3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67 not found: ID does not exist" Nov 04 04:47:37.875311 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.875285 4374 scope.go:122] "RemoveContainer" containerID="c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a" Nov 04 04:47:37.875528 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:47:37.875511 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a\": container with ID starting with c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a not found: ID does not exist" containerID="c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a" Nov 04 04:47:37.875566 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.875532 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a"} err="failed to get container status \"c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a\": rpc error: code = NotFound desc = could not find container \"c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a\": container with ID starting with c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a not found: ID does not exist" Nov 04 04:47:37.875566 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.875544 4374 scope.go:122] "RemoveContainer" containerID="d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509" Nov 04 04:47:37.875748 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:47:37.875732 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509\": container with ID starting with d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509 not found: ID does not exist" containerID="d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509" Nov 04 04:47:37.875809 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.875754 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509"} err="failed to get container status \"d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509\": rpc error: code = NotFound desc = could not find container \"d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509\": container with ID starting with d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509 not found: ID does not exist" Nov 04 04:47:37.875809 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.875774 4374 scope.go:122] "RemoveContainer" containerID="e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15" Nov 04 04:47:37.876027 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:47:37.876011 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15\": container with ID starting with e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15 not found: ID does not exist" containerID="e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15" Nov 04 04:47:37.876091 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.876033 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15"} err="failed to get container status \"e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15\": rpc error: code = NotFound desc = could not find container \"e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15\": container with ID starting with e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15 not found: ID does not exist" Nov 04 04:47:37.876091 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.876052 4374 scope.go:122] "RemoveContainer" containerID="021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858" Nov 04 04:47:37.876313 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:47:37.876288 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858\": container with ID starting with 021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858 not found: ID does not exist" containerID="021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858" Nov 04 04:47:37.876348 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.876319 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858"} err="failed to get container status \"021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858\": rpc error: code = NotFound desc = could not find container \"021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858\": container with ID starting with 021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858 not found: ID does not exist" Nov 04 04:47:37.876348 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.876334 4374 scope.go:122] "RemoveContainer" containerID="891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7" Nov 04 04:47:37.876579 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.876560 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7"} err="failed to get container status \"891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7\": rpc error: code = NotFound desc = could not find container \"891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7\": container with ID starting with 891a3f4b014caed1f26a29dce13ebf4ee07881598d74f45d7d43ac77863e9ad7 not found: ID does not exist" Nov 04 04:47:37.876625 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.876579 4374 scope.go:122] "RemoveContainer" containerID="ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c" Nov 04 04:47:37.876801 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.876785 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c"} err="failed to get container status \"ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c\": rpc error: code = NotFound desc = could not find container \"ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c\": container with ID starting with ca56030684d8dfecb0719c096196dd324c0715aa10414772ef295e978258be6c not found: ID does not exist" Nov 04 04:47:37.876840 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.876802 4374 scope.go:122] "RemoveContainer" containerID="3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67" Nov 04 04:47:37.877036 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.877019 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67"} err="failed to get container status \"3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67\": rpc error: code = NotFound desc = could not find container \"3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67\": container with ID starting with 3458e279824d678e4e2180d3e3f7e9d970d6f1202ff31184c987c693a8528c67 not found: ID does not exist" Nov 04 04:47:37.877082 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.877037 4374 scope.go:122] "RemoveContainer" containerID="c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a" Nov 04 04:47:37.877259 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.877242 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a"} err="failed to get container status \"c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a\": rpc error: code = NotFound desc = could not find container \"c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a\": container with ID starting with c7a0063acaf08dbc371ad4a53a9a6ab8a2375c3d584131412a56db1708680e1a not found: ID does not exist" Nov 04 04:47:37.877298 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.877260 4374 scope.go:122] "RemoveContainer" containerID="d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509" Nov 04 04:47:37.877465 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.877448 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509"} err="failed to get container status \"d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509\": rpc error: code = NotFound desc = could not find container \"d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509\": container with ID starting with d9c0add141e35944ccdd0c3b1832bbe40213abf175c609c616e4e477d29be509 not found: ID does not exist" Nov 04 04:47:37.877505 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.877466 4374 scope.go:122] "RemoveContainer" containerID="e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15" Nov 04 04:47:37.877683 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.877665 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15"} err="failed to get container status \"e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15\": rpc error: code = NotFound desc = could not find container \"e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15\": container with ID starting with e57583c2864d0f1488058f1886f8b6fc6b7e7e26d3cd85556c0d208ef77e9a15 not found: ID does not exist" Nov 04 04:47:37.877733 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.877683 4374 scope.go:122] "RemoveContainer" containerID="021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858" Nov 04 04:47:37.877928 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:37.877848 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858"} err="failed to get container status \"021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858\": rpc error: code = NotFound desc = could not find container \"021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858\": container with ID starting with 021eeca935ebe9f4c220428ec30d9afac6a6b2fe974afccbef692515802a4858 not found: ID does not exist" Nov 04 04:47:38.028468 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.028438 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 04:47:38.074954 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.074850 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 04:47:38.075538 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075517 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 04:47:38.075758 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075748 4374 cpu_manager.go:414] "containerMap: removing container" podUID="d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" containerName="console" Nov 04 04:47:38.075834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075760 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" containerName="console" Nov 04 04:47:38.075834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075777 4374 cpu_manager.go:414] "containerMap: removing container" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="kube-rbac-proxy-web" Nov 04 04:47:38.075834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075782 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="kube-rbac-proxy-web" Nov 04 04:47:38.075834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075789 4374 cpu_manager.go:414] "containerMap: removing container" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="kube-rbac-proxy" Nov 04 04:47:38.075834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075794 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="kube-rbac-proxy" Nov 04 04:47:38.075834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075805 4374 cpu_manager.go:414] "containerMap: removing container" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="alertmanager" Nov 04 04:47:38.075834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075810 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="alertmanager" Nov 04 04:47:38.075834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075818 4374 cpu_manager.go:414] "containerMap: removing container" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="config-reloader" Nov 04 04:47:38.075834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075823 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="config-reloader" Nov 04 04:47:38.075834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075829 4374 cpu_manager.go:414] "containerMap: removing container" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="kube-rbac-proxy-metric" Nov 04 04:47:38.075834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075835 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="kube-rbac-proxy-metric" Nov 04 04:47:38.076158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075844 4374 cpu_manager.go:414] "containerMap: removing container" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="init-config-reloader" Nov 04 04:47:38.076158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075849 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="init-config-reloader" Nov 04 04:47:38.076158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075859 4374 cpu_manager.go:414] "containerMap: removing container" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="prom-label-proxy" Nov 04 04:47:38.076158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075863 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="prom-label-proxy" Nov 04 04:47:38.076158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075919 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="config-reloader" Nov 04 04:47:38.076158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075927 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="kube-rbac-proxy-web" Nov 04 04:47:38.076158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075932 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="kube-rbac-proxy" Nov 04 04:47:38.076158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075938 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="kube-rbac-proxy-metric" Nov 04 04:47:38.076158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075943 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="prom-label-proxy" Nov 04 04:47:38.076158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075948 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="a009301a-821d-4baa-b6c2-877574019d0c" containerName="alertmanager" Nov 04 04:47:38.076158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.075954 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8edf2a1-5a5f-4ef6-bacf-b737a5733b5b" containerName="console" Nov 04 04:47:38.080842 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.080824 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.130033 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.129989 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 04 04:47:38.130033 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.129989 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 04 04:47:38.130033 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.129989 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 04 04:47:38.130286 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.129989 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 04 04:47:38.130286 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.130055 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 04 04:47:38.130286 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.130026 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 04 04:47:38.130286 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.130059 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 04 04:47:38.130286 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.130108 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-l4gmp\"" Nov 04 04:47:38.130562 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.130438 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 04 04:47:38.176383 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.176351 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 04 04:47:38.198857 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.198823 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ba467c78-8d75-4a2d-983d-427b4d983cca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.199024 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.198863 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98fx8\" (UniqueName: \"kubernetes.io/projected/ba467c78-8d75-4a2d-983d-427b4d983cca-kube-api-access-98fx8\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.199024 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.198950 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ba467c78-8d75-4a2d-983d-427b4d983cca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.199024 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.198977 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.199024 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.199006 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.199152 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.199076 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba467c78-8d75-4a2d-983d-427b4d983cca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.199152 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.199119 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.199213 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.199148 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.199213 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.199171 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-config-volume\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.199213 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.199190 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-web-config\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.199306 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.199210 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.199306 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.199255 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ba467c78-8d75-4a2d-983d-427b4d983cca-config-out\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.199367 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.199311 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba467c78-8d75-4a2d-983d-427b4d983cca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.299682 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299646 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-web-config\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.299682 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299683 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299703 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ba467c78-8d75-4a2d-983d-427b4d983cca-config-out\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299707 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299733 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba467c78-8d75-4a2d-983d-427b4d983cca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299772 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ba467c78-8d75-4a2d-983d-427b4d983cca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299806 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98fx8\" (UniqueName: \"kubernetes.io/projected/ba467c78-8d75-4a2d-983d-427b4d983cca-kube-api-access-98fx8\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299829 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ba467c78-8d75-4a2d-983d-427b4d983cca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299850 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299907 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299938 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba467c78-8d75-4a2d-983d-427b4d983cca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299965 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.299994 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.300022 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-config-volume\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.300467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.300246 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ba467c78-8d75-4a2d-983d-427b4d983cca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.302945 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.302914 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba467c78-8d75-4a2d-983d-427b4d983cca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.303166 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.303141 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-web-config\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.303456 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.303409 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba467c78-8d75-4a2d-983d-427b4d983cca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.303979 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.303938 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.304892 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.304827 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.305564 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.305530 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.305650 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.305617 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ba467c78-8d75-4a2d-983d-427b4d983cca-config-out\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.305705 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.305680 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.306294 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.306266 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.306721 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.306695 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ba467c78-8d75-4a2d-983d-427b4d983cca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.306808 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.306791 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ba467c78-8d75-4a2d-983d-427b4d983cca-config-volume\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.360318 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.360256 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98fx8\" (UniqueName: \"kubernetes.io/projected/ba467c78-8d75-4a2d-983d-427b4d983cca-kube-api-access-98fx8\") pod \"alertmanager-main-0\" (UID: \"ba467c78-8d75-4a2d-983d-427b4d983cca\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.405270 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.405235 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 04:47:38.410032 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.410012 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a009301a-821d-4baa-b6c2-877574019d0c" path="/var/lib/kubelet/pods/a009301a-821d-4baa-b6c2-877574019d0c/volumes" Nov 04 04:47:38.822160 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.822128 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 04:47:38.825174 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:47:38.825147 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba467c78_8d75_4a2d_983d_427b4d983cca.slice/crio-9a1089f8177a873e4252c92c03f9066e994ac5df74ef67ff7734f67c6efb3cba WatchSource:0}: Error finding container 9a1089f8177a873e4252c92c03f9066e994ac5df74ef67ff7734f67c6efb3cba: Status 404 returned error can't find the container with id 9a1089f8177a873e4252c92c03f9066e994ac5df74ef67ff7734f67c6efb3cba Nov 04 04:47:38.836360 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:38.836287 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ba467c78-8d75-4a2d-983d-427b4d983cca","Type":"ContainerStarted","Data":"9a1089f8177a873e4252c92c03f9066e994ac5df74ef67ff7734f67c6efb3cba"} Nov 04 04:47:39.232160 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:39.232129 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:39.372920 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:39.372891 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:39.841348 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:39.841315 4374 generic.go:356] "Generic (PLEG): container finished" podID="ba467c78-8d75-4a2d-983d-427b4d983cca" containerID="71a899036e0bae9096535c3648559e8d0b4e13a25558e0c20fb9a65822729b9f" exitCode=0 Nov 04 04:47:39.841746 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:39.841351 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ba467c78-8d75-4a2d-983d-427b4d983cca","Type":"ContainerDied","Data":"71a899036e0bae9096535c3648559e8d0b4e13a25558e0c20fb9a65822729b9f"} Nov 04 04:47:40.345447 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:40.345359 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:40.493012 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:40.492983 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:40.848575 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:40.848542 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ba467c78-8d75-4a2d-983d-427b4d983cca","Type":"ContainerStarted","Data":"a87a3ad386462c79c14649b85d204da2325a0c836717fde295e78b8ab4ae0fbb"} Nov 04 04:47:40.848982 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:40.848604 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ba467c78-8d75-4a2d-983d-427b4d983cca","Type":"ContainerStarted","Data":"be41d469ac5fb8599493318deb35bb5919b4a206a1528d1372c63389349fabc5"} Nov 04 04:47:40.848982 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:40.848633 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ba467c78-8d75-4a2d-983d-427b4d983cca","Type":"ContainerStarted","Data":"a8763f116d005680139952c0500f424ea8901b095816e4f70b519feb53524590"} Nov 04 04:47:40.848982 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:40.848657 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ba467c78-8d75-4a2d-983d-427b4d983cca","Type":"ContainerStarted","Data":"1b9beb4a5e830319a87ae09e1578405dbc3e915ec9fa142cc4944f3f87972af8"} Nov 04 04:47:40.848982 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:40.848679 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ba467c78-8d75-4a2d-983d-427b4d983cca","Type":"ContainerStarted","Data":"98bd049465815ceafbc13ef4b4ff651f05fa11ab388025f715b75186a846ed1e"} Nov 04 04:47:40.848982 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:40.848705 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ba467c78-8d75-4a2d-983d-427b4d983cca","Type":"ContainerStarted","Data":"8684c03d6e596fd35f468c58754c083bff624349a5a21ac82ab43e8b96bc8499"} Nov 04 04:47:40.971151 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:40.971105 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.958923301 podStartE2EDuration="2.971091288s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.167987ms" isStatefulPod=false podCreationTimestamp="2026-11-04 04:47:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:47:40.969637712 +0000 UTC m=+102.681328873" watchObservedRunningTime="2026-11-04 04:47:40.971091288 +0000 UTC m=+102.682782428" Nov 04 04:47:43.596060 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.596020 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-f86654974-6qxm7"] Nov 04 04:47:43.599471 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.599442 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.648705 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.648684 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Nov 04 04:47:43.648830 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.648723 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Nov 04 04:47:43.695026 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.694994 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Nov 04 04:47:43.695205 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.695110 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Nov 04 04:47:43.695205 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.695111 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Nov 04 04:47:43.695205 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.695117 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Nov 04 04:47:43.695205 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.695162 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Nov 04 04:47:43.741242 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.741218 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7pqf8\"" Nov 04 04:47:43.744480 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.744457 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms29m\" (UniqueName: \"kubernetes.io/projected/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-kube-api-access-ms29m\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.744576 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.744498 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-console-oauth-config\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.744576 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.744530 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-service-ca\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.744685 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.744664 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-tmp\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.744740 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.744720 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-console-serving-cert\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.744790 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.744774 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-console-config\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.744844 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.744826 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-trusted-ca-bundle\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.744911 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.744857 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-oauth-serving-cert\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.788668 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.788639 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Nov 04 04:47:43.789708 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.789687 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f86654974-6qxm7"] Nov 04 04:47:43.845768 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.845740 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-console-config\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.845887 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.845780 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-trusted-ca-bundle\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.845887 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.845807 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-oauth-serving-cert\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.845887 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.845843 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms29m\" (UniqueName: \"kubernetes.io/projected/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-kube-api-access-ms29m\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.846026 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.845904 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-console-oauth-config\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.846026 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.845936 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-service-ca\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.846026 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.845966 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-tmp\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.846026 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.845991 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-console-serving-cert\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.846628 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.846575 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-tmp\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.846726 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.846602 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-console-config\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.846787 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.846738 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-trusted-ca-bundle\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.846832 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.846799 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-oauth-serving-cert\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.846990 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.846967 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-service-ca\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.848578 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.848555 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-console-oauth-config\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.848711 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.848697 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-console-serving-cert\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.856992 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.856967 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:43.902491 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.902473 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms29m\" (UniqueName: \"kubernetes.io/projected/ece0eaf0-b2e1-4ab1-b563-4ed302ae0749-kube-api-access-ms29m\") pod \"console-f86654974-6qxm7\" (UID: \"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749\") " pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:43.911342 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:43.911320 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:44.227800 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:44.227769 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:44.273258 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:44.273230 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f86654974-6qxm7"] Nov 04 04:47:44.276204 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:47:44.276166 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece0eaf0_b2e1_4ab1_b563_4ed302ae0749.slice/crio-cfcf15428f0e203285871694dd20115610b753cc575c3bd1af18e7e1d4eb4975 WatchSource:0}: Error finding container cfcf15428f0e203285871694dd20115610b753cc575c3bd1af18e7e1d4eb4975: Status 404 returned error can't find the container with id cfcf15428f0e203285871694dd20115610b753cc575c3bd1af18e7e1d4eb4975 Nov 04 04:47:44.369299 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:44.369265 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:44.863999 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:44.863965 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f86654974-6qxm7" event={"ID":"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749","Type":"ContainerStarted","Data":"19bfb6e0070517060eb7c678a828624eb0fab82454410cc7d905f019165462af"} Nov 04 04:47:44.864461 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:44.864020 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f86654974-6qxm7" event={"ID":"ece0eaf0-b2e1-4ab1-b563-4ed302ae0749","Type":"ContainerStarted","Data":"cfcf15428f0e203285871694dd20115610b753cc575c3bd1af18e7e1d4eb4975"} Nov 04 04:47:44.974566 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:44.974514 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-f86654974-6qxm7" podStartSLOduration=1.974500573 podStartE2EDuration="1.974500573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:47:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:47:44.973380172 +0000 UTC m=+106.685071312" watchObservedRunningTime="2026-11-04 04:47:44.974500573 +0000 UTC m=+106.686191711" Nov 04 04:47:47.960157 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:47.960121 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:48.101944 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:48.101910 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:47:53.911455 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:53.911419 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:53.911914 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:53.911475 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:53.916262 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:53.916240 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:54.902276 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:47:54.902245 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f86654974-6qxm7" Nov 04 04:47:55.831410 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:47:55.831368 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:48:25.831362 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:48:25.831283 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:48:39.105737 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:48:39.105705 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:48:39.585379 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:48:39.585348 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:48:55.831320 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:48:55.831283 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:49:10.423526 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:10.423485 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:10.591854 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:10.591813 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:10.735128 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:10.735096 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:11.890175 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:11.890141 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88"] Nov 04 04:49:11.894004 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:11.893979 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:11.989544 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:11.989519 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Nov 04 04:49:11.989544 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:11.989547 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Nov 04 04:49:11.989942 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:11.989752 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-87jsw\"" Nov 04 04:49:11.990838 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:11.990809 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88"] Nov 04 04:49:11.997085 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:11.997066 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88\" (UID: \"191e7f44-4617-422d-a62d-83896dbedd18\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:11.997167 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:11.997098 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbhqd\" (UniqueName: \"kubernetes.io/projected/191e7f44-4617-422d-a62d-83896dbedd18-kube-api-access-vbhqd\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88\" (UID: \"191e7f44-4617-422d-a62d-83896dbedd18\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:11.997167 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:11.997131 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88\" (UID: \"191e7f44-4617-422d-a62d-83896dbedd18\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:12.032147 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.032102 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:12.038411 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.038388 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4"] Nov 04 04:49:12.041396 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.041382 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.091694 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.091670 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Nov 04 04:49:12.091694 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.091715 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Nov 04 04:49:12.092215 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.092178 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Nov 04 04:49:12.092307 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.092291 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Nov 04 04:49:12.092409 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.092314 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Nov 04 04:49:12.092409 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.092311 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Nov 04 04:49:12.092409 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.092380 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zd5b6\"" Nov 04 04:49:12.093088 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.093064 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4"] Nov 04 04:49:12.097714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.097691 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88\" (UID: \"191e7f44-4617-422d-a62d-83896dbedd18\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:12.097817 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.097730 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbhqd\" (UniqueName: \"kubernetes.io/projected/191e7f44-4617-422d-a62d-83896dbedd18-kube-api-access-vbhqd\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88\" (UID: \"191e7f44-4617-422d-a62d-83896dbedd18\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:12.097817 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.097770 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88\" (UID: \"191e7f44-4617-422d-a62d-83896dbedd18\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:12.098138 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.098119 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88\" (UID: \"191e7f44-4617-422d-a62d-83896dbedd18\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:12.098311 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.098291 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88\" (UID: \"191e7f44-4617-422d-a62d-83896dbedd18\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:12.155539 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.155484 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbhqd\" (UniqueName: \"kubernetes.io/projected/191e7f44-4617-422d-a62d-83896dbedd18-kube-api-access-vbhqd\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88\" (UID: \"191e7f44-4617-422d-a62d-83896dbedd18\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:12.184192 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.184170 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:12.198269 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.198251 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.198332 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.198281 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.198332 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.198296 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmrwf\" (UniqueName: \"kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-kube-api-access-mmrwf\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.198332 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.198314 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.198441 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.198429 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.198479 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.198451 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.198515 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.198475 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/22b15e41-5032-490c-a660-0bc50e3d4750-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.210238 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.210222 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:12.299345 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.299317 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/22b15e41-5032-490c-a660-0bc50e3d4750-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.299495 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.299359 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.299495 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.299387 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.299495 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.299403 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmrwf\" (UniqueName: \"kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-kube-api-access-mmrwf\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.299495 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.299429 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.299495 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.299473 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.299757 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.299501 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.300112 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.300084 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.301472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.301442 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/22b15e41-5032-490c-a660-0bc50e3d4750-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.301715 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.301698 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.302108 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.302091 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.302185 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.302115 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.364021 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.363998 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.369386 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.369363 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmrwf\" (UniqueName: \"kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-kube-api-access-mmrwf\") pod \"istiod-openshift-gateway-7876bfcb74-xc7p4\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:12.570162 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.570134 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88"] Nov 04 04:49:12.572850 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:49:12.572825 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191e7f44_4617_422d_a62d_83896dbedd18.slice/crio-504e0b24586127b04cdeb76a2814bae280050d85a7a7900d4e79b4b44c388dc2 WatchSource:0}: Error finding container 504e0b24586127b04cdeb76a2814bae280050d85a7a7900d4e79b4b44c388dc2: Status 404 returned error can't find the container with id 504e0b24586127b04cdeb76a2814bae280050d85a7a7900d4e79b4b44c388dc2 Nov 04 04:49:12.653137 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:12.653104 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:13.064760 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:13.064730 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4"] Nov 04 04:49:13.067863 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:49:13.067835 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22b15e41_5032_490c_a660_0bc50e3d4750.slice/crio-83bdb3c1e91383b305f22e38909f71d3f3681e00a8b9eb120d1dd5dd5f28ffda WatchSource:0}: Error finding container 83bdb3c1e91383b305f22e38909f71d3f3681e00a8b9eb120d1dd5dd5f28ffda: Status 404 returned error can't find the container with id 83bdb3c1e91383b305f22e38909f71d3f3681e00a8b9eb120d1dd5dd5f28ffda Nov 04 04:49:13.151223 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:13.151190 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" event={"ID":"22b15e41-5032-490c-a660-0bc50e3d4750","Type":"ContainerStarted","Data":"83bdb3c1e91383b305f22e38909f71d3f3681e00a8b9eb120d1dd5dd5f28ffda"} Nov 04 04:49:13.152308 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:13.152275 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" event={"ID":"191e7f44-4617-422d-a62d-83896dbedd18","Type":"ContainerStarted","Data":"504e0b24586127b04cdeb76a2814bae280050d85a7a7900d4e79b4b44c388dc2"} Nov 04 04:49:13.526649 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:13.526616 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:17.549177 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:17.549143 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:49:17.549541 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:17.549216 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:49:18.177977 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:18.177938 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" event={"ID":"22b15e41-5032-490c-a660-0bc50e3d4750","Type":"ContainerStarted","Data":"6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb"} Nov 04 04:49:18.178201 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:18.178181 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:18.180155 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:18.180120 4374 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-xc7p4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 04:49:18.180301 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:18.180189 4374 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" podUID="22b15e41-5032-490c-a660-0bc50e3d4750" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 04:49:18.371734 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:18.371672 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" podStartSLOduration=2.893094106 podStartE2EDuration="7.371653484s" totalImagesPullingTime="4.478559378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:49:18.370800081 +0000 UTC m=+200.082491221" watchObservedRunningTime="2026-11-04 04:49:18.371653484 +0000 UTC m=+200.083344626" Nov 04 04:49:19.046290 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:19.046254 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:19.183178 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:19.183145 4374 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-xc7p4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 04:49:19.183343 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:19.183217 4374 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" podUID="22b15e41-5032-490c-a660-0bc50e3d4750" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 04:49:19.188691 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:19.188652 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:19.542110 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:19.542082 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:19.865369 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:19.865338 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:20.010988 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:20.010959 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:20.187491 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:20.187400 4374 generic.go:356] "Generic (PLEG): container finished" podID="191e7f44-4617-422d-a62d-83896dbedd18" containerID="447f413d677e1780d7362ec3f3c611d8784d71045380d6e0e830979629a18983" exitCode=0 Nov 04 04:49:20.187491 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:20.187449 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" event={"ID":"191e7f44-4617-422d-a62d-83896dbedd18","Type":"ContainerDied","Data":"447f413d677e1780d7362ec3f3c611d8784d71045380d6e0e830979629a18983"} Nov 04 04:49:22.183339 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:22.183310 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:49:22.195151 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:22.195123 4374 generic.go:356] "Generic (PLEG): container finished" podID="191e7f44-4617-422d-a62d-83896dbedd18" containerID="5963b637d14706815e1a075f2bcd6b86cb54c8116ed2b590a633e60dabb5b218" exitCode=0 Nov 04 04:49:22.195283 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:22.195184 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" event={"ID":"191e7f44-4617-422d-a62d-83896dbedd18","Type":"ContainerDied","Data":"5963b637d14706815e1a075f2bcd6b86cb54c8116ed2b590a633e60dabb5b218"} Nov 04 04:49:23.613405 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:23.613372 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:25.836680 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:49:25.836644 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:49:30.234688 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:30.234653 4374 generic.go:356] "Generic (PLEG): container finished" podID="191e7f44-4617-422d-a62d-83896dbedd18" containerID="6aa53bbbe2ab0b1c99d0f18bbe4ca9d5878e6848c10261f618be72d1a8a96544" exitCode=0 Nov 04 04:49:30.235086 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:30.234701 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" event={"ID":"191e7f44-4617-422d-a62d-83896dbedd18","Type":"ContainerDied","Data":"6aa53bbbe2ab0b1c99d0f18bbe4ca9d5878e6848c10261f618be72d1a8a96544"} Nov 04 04:49:31.364816 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:31.364792 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:31.564226 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:31.564160 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-util\" (UniqueName: \"kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-util\") pod \"191e7f44-4617-422d-a62d-83896dbedd18\" (UID: \"191e7f44-4617-422d-a62d-83896dbedd18\") " Nov 04 04:49:31.564226 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:31.564191 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-bundle\" (UniqueName: \"kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-bundle\") pod \"191e7f44-4617-422d-a62d-83896dbedd18\" (UID: \"191e7f44-4617-422d-a62d-83896dbedd18\") " Nov 04 04:49:31.564466 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:31.564268 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/191e7f44-4617-422d-a62d-83896dbedd18-kube-api-access-vbhqd\" (UniqueName: \"kubernetes.io/projected/191e7f44-4617-422d-a62d-83896dbedd18-kube-api-access-vbhqd\") pod \"191e7f44-4617-422d-a62d-83896dbedd18\" (UID: \"191e7f44-4617-422d-a62d-83896dbedd18\") " Nov 04 04:49:31.564864 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:31.564835 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-bundle" pod "191e7f44-4617-422d-a62d-83896dbedd18" (UID: "191e7f44-4617-422d-a62d-83896dbedd18"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:49:31.566319 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:31.566290 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/191e7f44-4617-422d-a62d-83896dbedd18-kube-api-access-vbhqd" pod "191e7f44-4617-422d-a62d-83896dbedd18" (UID: "191e7f44-4617-422d-a62d-83896dbedd18"). InnerVolumeSpecName "kube-api-access-vbhqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:49:31.569172 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:31.569152 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-util" pod "191e7f44-4617-422d-a62d-83896dbedd18" (UID: "191e7f44-4617-422d-a62d-83896dbedd18"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:49:31.665423 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:31.665384 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbhqd\" (UniqueName: \"kubernetes.io/projected/191e7f44-4617-422d-a62d-83896dbedd18-kube-api-access-vbhqd\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:49:31.665423 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:31.665420 4374 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-util\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:49:31.665423 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:31.665430 4374 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/191e7f44-4617-422d-a62d-83896dbedd18-bundle\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:49:32.243377 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:32.243343 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" event={"ID":"191e7f44-4617-422d-a62d-83896dbedd18","Type":"ContainerDied","Data":"504e0b24586127b04cdeb76a2814bae280050d85a7a7900d4e79b4b44c388dc2"} Nov 04 04:49:32.243538 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:32.243390 4374 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="504e0b24586127b04cdeb76a2814bae280050d85a7a7900d4e79b4b44c388dc2" Nov 04 04:49:32.243538 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:32.243353 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71zcf88" Nov 04 04:49:39.953994 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:39.953921 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:50.966630 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:50.966589 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:51.242898 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.242852 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:51.642902 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.642824 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:51.898378 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.898284 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc"] Nov 04 04:49:51.898965 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.898945 4374 cpu_manager.go:414] "containerMap: removing container" podUID="191e7f44-4617-422d-a62d-83896dbedd18" containerName="pull" Nov 04 04:49:51.899061 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.898966 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="191e7f44-4617-422d-a62d-83896dbedd18" containerName="pull" Nov 04 04:49:51.899061 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.898994 4374 cpu_manager.go:414] "containerMap: removing container" podUID="191e7f44-4617-422d-a62d-83896dbedd18" containerName="extract" Nov 04 04:49:51.899061 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.899002 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="191e7f44-4617-422d-a62d-83896dbedd18" containerName="extract" Nov 04 04:49:51.899061 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.899040 4374 cpu_manager.go:414] "containerMap: removing container" podUID="191e7f44-4617-422d-a62d-83896dbedd18" containerName="util" Nov 04 04:49:51.899061 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.899047 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="191e7f44-4617-422d-a62d-83896dbedd18" containerName="util" Nov 04 04:49:51.899286 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.899178 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="191e7f44-4617-422d-a62d-83896dbedd18" containerName="extract" Nov 04 04:49:51.903332 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.903295 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:51.967063 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.967034 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Nov 04 04:49:51.967383 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.967081 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Nov 04 04:49:51.967383 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.967098 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Nov 04 04:49:51.967383 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.967250 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-jhscp\"" Nov 04 04:49:51.986132 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:51.986095 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:52.018761 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.018732 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a26af025-7313-4e3c-b7c4-ddf3ab141b92-apiservice-cert\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.019054 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.018771 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a26af025-7313-4e3c-b7c4-ddf3ab141b92-metrics-cert\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.019054 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.018806 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a26af025-7313-4e3c-b7c4-ddf3ab141b92-manifests\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.019054 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.018837 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdjfq\" (UniqueName: \"kubernetes.io/projected/a26af025-7313-4e3c-b7c4-ddf3ab141b92-kube-api-access-zdjfq\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.019054 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.018855 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a26af025-7313-4e3c-b7c4-ddf3ab141b92-tmp\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.019054 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.018913 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a26af025-7313-4e3c-b7c4-ddf3ab141b92-webhook-cert\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.033400 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.033379 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Nov 04 04:49:52.033502 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.033426 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Nov 04 04:49:52.046902 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.046861 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc"] Nov 04 04:49:52.119976 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.119945 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a26af025-7313-4e3c-b7c4-ddf3ab141b92-webhook-cert\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.120151 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.119992 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a26af025-7313-4e3c-b7c4-ddf3ab141b92-apiservice-cert\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.120151 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.120013 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a26af025-7313-4e3c-b7c4-ddf3ab141b92-metrics-cert\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.120151 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.120071 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a26af025-7313-4e3c-b7c4-ddf3ab141b92-manifests\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.120151 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.120122 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdjfq\" (UniqueName: \"kubernetes.io/projected/a26af025-7313-4e3c-b7c4-ddf3ab141b92-kube-api-access-zdjfq\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.120362 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.120154 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a26af025-7313-4e3c-b7c4-ddf3ab141b92-tmp\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.120642 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.120610 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a26af025-7313-4e3c-b7c4-ddf3ab141b92-manifests\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.122416 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.122386 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a26af025-7313-4e3c-b7c4-ddf3ab141b92-tmp\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.122526 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.122489 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a26af025-7313-4e3c-b7c4-ddf3ab141b92-webhook-cert\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.122922 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.122901 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a26af025-7313-4e3c-b7c4-ddf3ab141b92-metrics-cert\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.122976 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.122910 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a26af025-7313-4e3c-b7c4-ddf3ab141b92-apiservice-cert\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.140293 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.140262 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:52.182084 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.182021 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdjfq\" (UniqueName: \"kubernetes.io/projected/a26af025-7313-4e3c-b7c4-ddf3ab141b92-kube-api-access-zdjfq\") pod \"opendatahub-operator-controller-manager-546b6bbf6-j2trc\" (UID: \"a26af025-7313-4e3c-b7c4-ddf3ab141b92\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.217634 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.217603 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:49:52.298857 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.298827 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:52.690316 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:52.690282 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc"] Nov 04 04:49:52.694697 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:49:52.694660 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda26af025_7313_4e3c_b7c4_ddf3ab141b92.slice/crio-ccd005113154a86c560633f3bd0d9a90893d1cc5015c87373dd2965f99a1419a WatchSource:0}: Error finding container ccd005113154a86c560633f3bd0d9a90893d1cc5015c87373dd2965f99a1419a: Status 404 returned error can't find the container with id ccd005113154a86c560633f3bd0d9a90893d1cc5015c87373dd2965f99a1419a Nov 04 04:49:53.317171 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:53.317138 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" event={"ID":"a26af025-7313-4e3c-b7c4-ddf3ab141b92","Type":"ContainerStarted","Data":"ccd005113154a86c560633f3bd0d9a90893d1cc5015c87373dd2965f99a1419a"} Nov 04 04:49:54.607854 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:54.607819 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:54.750879 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:54.750838 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:54.970651 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:54.970577 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:55.116780 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:55.116740 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:49:55.836088 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:49:55.836044 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:49:57.334856 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:57.334823 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" event={"ID":"a26af025-7313-4e3c-b7c4-ddf3ab141b92","Type":"ContainerStarted","Data":"47c5c8f999a21ee3851fdcc1a241a3c9f5f226b6472ef3013a7ec8ab4b8a9c25"} Nov 04 04:49:58.340018 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:58.339982 4374 generic.go:356] "Generic (PLEG): container finished" podID="a26af025-7313-4e3c-b7c4-ddf3ab141b92" containerID="47c5c8f999a21ee3851fdcc1a241a3c9f5f226b6472ef3013a7ec8ab4b8a9c25" exitCode=0 Nov 04 04:49:58.340018 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:49:58.340020 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" event={"ID":"a26af025-7313-4e3c-b7c4-ddf3ab141b92","Type":"ContainerDied","Data":"47c5c8f999a21ee3851fdcc1a241a3c9f5f226b6472ef3013a7ec8ab4b8a9c25"} Nov 04 04:50:00.349011 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:00.348974 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" event={"ID":"a26af025-7313-4e3c-b7c4-ddf3ab141b92","Type":"ContainerStarted","Data":"401f6e444041e8f5e6299ef92acc95fc6034652fb4e82ed17e01b0abdcbe69df"} Nov 04 04:50:00.349435 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:00.349119 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:50:00.600017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:00.599939 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:00.619446 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:00.619400 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" podStartSLOduration=3.618080964 podStartE2EDuration="9.619387115s" totalImagesPullingTime="5.540634019s" totalInitContainerRuntime="460.672132ms" isStatefulPod=false podCreationTimestamp="2026-11-04 04:49:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:50:00.572094671 +0000 UTC m=+242.283785845" watchObservedRunningTime="2026-11-04 04:50:00.619387115 +0000 UTC m=+242.331078298" Nov 04 04:50:00.748817 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:00.748791 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:02.350225 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:02.350185 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:04.750879 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:04.750828 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:05.955096 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:05.955062 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:07.161632 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:07.161604 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:07.550480 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:07.550446 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:11.354981 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:11.354949 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-546b6bbf6-j2trc" Nov 04 04:50:13.609535 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:13.609500 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:14.554389 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:14.554354 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:16.561267 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:16.561235 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:16.775354 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:16.775324 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:17.058462 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:17.058425 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:17.718132 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:17.718095 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:17.868318 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:17.868286 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:18.201204 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:18.201173 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:19.153750 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:19.153721 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:19.767237 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:19.767196 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:20.190041 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:20.189958 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:23.433435 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:23.433396 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:23.592687 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:23.592649 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:24.002915 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:24.002883 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:24.442410 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:24.442329 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:24.611786 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:24.611752 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:25.604205 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.604167 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:25.691491 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.691459 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x"] Nov 04 04:50:25.694695 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.694676 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" Nov 04 04:50:25.768446 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.768399 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:25.784128 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.784096 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x"] Nov 04 04:50:25.820542 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.820509 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4v7d\" (UniqueName: \"kubernetes.io/projected/321d39ea-e4bc-4cdf-9cf6-015d7c191dcc-kube-api-access-b4v7d\") pod \"kube-auth-proxy-655d896fb8-b9v6x\" (UID: \"321d39ea-e4bc-4cdf-9cf6-015d7c191dcc\") " pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" Nov 04 04:50:25.820711 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.820557 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/321d39ea-e4bc-4cdf-9cf6-015d7c191dcc-tmp\") pod \"kube-auth-proxy-655d896fb8-b9v6x\" (UID: \"321d39ea-e4bc-4cdf-9cf6-015d7c191dcc\") " pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" Nov 04 04:50:25.820779 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.820707 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/321d39ea-e4bc-4cdf-9cf6-015d7c191dcc-tls-certs\") pod \"kube-auth-proxy-655d896fb8-b9v6x\" (UID: \"321d39ea-e4bc-4cdf-9cf6-015d7c191dcc\") " pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" Nov 04 04:50:25.837907 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.837857 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Nov 04 04:50:25.838065 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:50:25.838005 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:50:25.838118 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.838095 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-txlxj\"" Nov 04 04:50:25.838162 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.838128 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Nov 04 04:50:25.921831 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.921767 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/321d39ea-e4bc-4cdf-9cf6-015d7c191dcc-tls-certs\") pod \"kube-auth-proxy-655d896fb8-b9v6x\" (UID: \"321d39ea-e4bc-4cdf-9cf6-015d7c191dcc\") " pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" Nov 04 04:50:25.921831 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.921819 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4v7d\" (UniqueName: \"kubernetes.io/projected/321d39ea-e4bc-4cdf-9cf6-015d7c191dcc-kube-api-access-b4v7d\") pod \"kube-auth-proxy-655d896fb8-b9v6x\" (UID: \"321d39ea-e4bc-4cdf-9cf6-015d7c191dcc\") " pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" Nov 04 04:50:25.922019 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.921887 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/321d39ea-e4bc-4cdf-9cf6-015d7c191dcc-tmp\") pod \"kube-auth-proxy-655d896fb8-b9v6x\" (UID: \"321d39ea-e4bc-4cdf-9cf6-015d7c191dcc\") " pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" Nov 04 04:50:25.924210 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.924186 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/321d39ea-e4bc-4cdf-9cf6-015d7c191dcc-tmp\") pod \"kube-auth-proxy-655d896fb8-b9v6x\" (UID: \"321d39ea-e4bc-4cdf-9cf6-015d7c191dcc\") " pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" Nov 04 04:50:25.924653 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.924632 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/321d39ea-e4bc-4cdf-9cf6-015d7c191dcc-tls-certs\") pod \"kube-auth-proxy-655d896fb8-b9v6x\" (UID: \"321d39ea-e4bc-4cdf-9cf6-015d7c191dcc\") " pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" Nov 04 04:50:25.980160 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.980130 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4v7d\" (UniqueName: \"kubernetes.io/projected/321d39ea-e4bc-4cdf-9cf6-015d7c191dcc-kube-api-access-b4v7d\") pod \"kube-auth-proxy-655d896fb8-b9v6x\" (UID: \"321d39ea-e4bc-4cdf-9cf6-015d7c191dcc\") " pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" Nov 04 04:50:25.981297 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:25.981237 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:26.010201 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:26.010173 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" Nov 04 04:50:26.186253 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:26.186162 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:26.340004 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:26.339954 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:26.371581 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:26.371549 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x"] Nov 04 04:50:26.375294 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:50:26.375269 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321d39ea_e4bc_4cdf_9cf6_015d7c191dcc.slice/crio-5f3eb3ee12bec79a92481e562b103a8e0eea50072fdef0af6eaf5677ec504457 WatchSource:0}: Error finding container 5f3eb3ee12bec79a92481e562b103a8e0eea50072fdef0af6eaf5677ec504457: Status 404 returned error can't find the container with id 5f3eb3ee12bec79a92481e562b103a8e0eea50072fdef0af6eaf5677ec504457 Nov 04 04:50:26.440057 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:26.439992 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" event={"ID":"321d39ea-e4bc-4cdf-9cf6-015d7c191dcc","Type":"ContainerStarted","Data":"5f3eb3ee12bec79a92481e562b103a8e0eea50072fdef0af6eaf5677ec504457"} Nov 04 04:50:26.485825 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:26.485796 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:26.631212 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:26.631181 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:27.918649 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:27.918613 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:28.063346 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:28.063306 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:32.461846 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:32.461812 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" event={"ID":"321d39ea-e4bc-4cdf-9cf6-015d7c191dcc","Type":"ContainerStarted","Data":"8634a5201abf8354359a13a05184976e37353e885bcad725d525e97eca2bce46"} Nov 04 04:50:32.573320 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:32.573269 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-655d896fb8-b9v6x" podStartSLOduration=2.410369431 podStartE2EDuration="7.573255025s" totalImagesPullingTime="5.162885594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:50:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:50:32.571924718 +0000 UTC m=+274.283615857" watchObservedRunningTime="2026-11-04 04:50:32.573255025 +0000 UTC m=+274.284946164" Nov 04 04:50:32.666040 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:32.666012 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:32.824444 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:32.824405 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:33.740920 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:33.740886 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:36.199726 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:36.199694 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:37.368442 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:37.368409 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:37.512197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:37.512158 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:38.686994 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.686964 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk"] Nov 04 04:50:38.690242 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.690220 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:38.729317 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.729289 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44df49b9-c08f-4cc7-8d90-fe3859a7585d-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-mxvsk\" (UID: \"44df49b9-c08f-4cc7-8d90-fe3859a7585d\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:38.729418 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.729322 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/44df49b9-c08f-4cc7-8d90-fe3859a7585d-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-mxvsk\" (UID: \"44df49b9-c08f-4cc7-8d90-fe3859a7585d\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:38.729418 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.729348 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6ktb\" (UniqueName: \"kubernetes.io/projected/44df49b9-c08f-4cc7-8d90-fe3859a7585d-kube-api-access-c6ktb\") pod \"kserve-module-controller-manager-7566cf5fc8-mxvsk\" (UID: \"44df49b9-c08f-4cc7-8d90-fe3859a7585d\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:38.729530 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.729509 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44df49b9-c08f-4cc7-8d90-fe3859a7585d-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-mxvsk\" (UID: \"44df49b9-c08f-4cc7-8d90-fe3859a7585d\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:38.785632 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.785605 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-9b4t8\"" Nov 04 04:50:38.785928 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.785905 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Nov 04 04:50:38.786059 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.786008 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Nov 04 04:50:38.830856 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.830825 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44df49b9-c08f-4cc7-8d90-fe3859a7585d-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-mxvsk\" (UID: \"44df49b9-c08f-4cc7-8d90-fe3859a7585d\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:38.830995 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.830883 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44df49b9-c08f-4cc7-8d90-fe3859a7585d-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-mxvsk\" (UID: \"44df49b9-c08f-4cc7-8d90-fe3859a7585d\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:38.830995 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.830916 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/44df49b9-c08f-4cc7-8d90-fe3859a7585d-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-mxvsk\" (UID: \"44df49b9-c08f-4cc7-8d90-fe3859a7585d\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:38.830995 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.830955 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6ktb\" (UniqueName: \"kubernetes.io/projected/44df49b9-c08f-4cc7-8d90-fe3859a7585d-kube-api-access-c6ktb\") pod \"kserve-module-controller-manager-7566cf5fc8-mxvsk\" (UID: \"44df49b9-c08f-4cc7-8d90-fe3859a7585d\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:38.831280 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.831263 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/44df49b9-c08f-4cc7-8d90-fe3859a7585d-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-mxvsk\" (UID: \"44df49b9-c08f-4cc7-8d90-fe3859a7585d\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:38.831439 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.831423 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44df49b9-c08f-4cc7-8d90-fe3859a7585d-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-mxvsk\" (UID: \"44df49b9-c08f-4cc7-8d90-fe3859a7585d\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:38.879552 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.879522 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" Nov 04 04:50:38.881189 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.881168 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk"] Nov 04 04:50:38.881864 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.881838 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44df49b9-c08f-4cc7-8d90-fe3859a7585d-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-mxvsk\" (UID: \"44df49b9-c08f-4cc7-8d90-fe3859a7585d\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:38.890536 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:38.890515 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6ktb\" (UniqueName: \"kubernetes.io/projected/44df49b9-c08f-4cc7-8d90-fe3859a7585d-kube-api-access-c6ktb\") pod \"kserve-module-controller-manager-7566cf5fc8-mxvsk\" (UID: \"44df49b9-c08f-4cc7-8d90-fe3859a7585d\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:39.002692 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:39.002667 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:39.414666 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:39.414592 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk"] Nov 04 04:50:39.417929 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:50:39.417901 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44df49b9_c08f_4cc7_8d90_fe3859a7585d.slice/crio-05dd4643c308243c97482ebb20cb8aa94c63174fdcbf523ef7d4f03c4ab21176 WatchSource:0}: Error finding container 05dd4643c308243c97482ebb20cb8aa94c63174fdcbf523ef7d4f03c4ab21176: Status 404 returned error can't find the container with id 05dd4643c308243c97482ebb20cb8aa94c63174fdcbf523ef7d4f03c4ab21176 Nov 04 04:50:39.488308 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:39.488278 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" event={"ID":"44df49b9-c08f-4cc7-8d90-fe3859a7585d","Type":"ContainerStarted","Data":"05dd4643c308243c97482ebb20cb8aa94c63174fdcbf523ef7d4f03c4ab21176"} Nov 04 04:50:40.312399 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:40.312364 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:42.233959 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:42.233919 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:42.379356 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:42.379328 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:43.512800 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:43.512760 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" event={"ID":"44df49b9-c08f-4cc7-8d90-fe3859a7585d","Type":"ContainerStarted","Data":"35b9b30674890aa468636ef0116700745a32270fe146f0635245c600382532c1"} Nov 04 04:50:43.513216 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:43.512932 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:50:43.620015 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:43.619985 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:43.673364 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:43.673309 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" podStartSLOduration=2.2224037 podStartE2EDuration="5.673294444s" totalImagesPullingTime="3.450890744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:50:43.67238786 +0000 UTC m=+285.384079034" watchObservedRunningTime="2026-11-04 04:50:43.673294444 +0000 UTC m=+285.384985583" Nov 04 04:50:46.093357 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.093312 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:50:46.096691 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.096670 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" Nov 04 04:50:46.190424 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.190390 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4a069916-bc79-42d8-8b30-be79dde85d61-config-file\") pod \"limitador-limitador-7c9b87d574-r5g6w\" (UID: \"4a069916-bc79-42d8-8b30-be79dde85d61\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" Nov 04 04:50:46.190596 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.190454 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg5qb\" (UniqueName: \"kubernetes.io/projected/4a069916-bc79-42d8-8b30-be79dde85d61-kube-api-access-qg5qb\") pod \"limitador-limitador-7c9b87d574-r5g6w\" (UID: \"4a069916-bc79-42d8-8b30-be79dde85d61\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" Nov 04 04:50:46.191829 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.191806 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Nov 04 04:50:46.191908 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.191827 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Nov 04 04:50:46.191908 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.191818 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-snt8z\"" Nov 04 04:50:46.191908 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.191831 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Nov 04 04:50:46.193267 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.193248 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:50:46.291046 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.291015 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg5qb\" (UniqueName: \"kubernetes.io/projected/4a069916-bc79-42d8-8b30-be79dde85d61-kube-api-access-qg5qb\") pod \"limitador-limitador-7c9b87d574-r5g6w\" (UID: \"4a069916-bc79-42d8-8b30-be79dde85d61\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" Nov 04 04:50:46.291220 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.291060 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4a069916-bc79-42d8-8b30-be79dde85d61-config-file\") pod \"limitador-limitador-7c9b87d574-r5g6w\" (UID: \"4a069916-bc79-42d8-8b30-be79dde85d61\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" Nov 04 04:50:46.291769 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.291749 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4a069916-bc79-42d8-8b30-be79dde85d61-config-file\") pod \"limitador-limitador-7c9b87d574-r5g6w\" (UID: \"4a069916-bc79-42d8-8b30-be79dde85d61\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" Nov 04 04:50:46.351418 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.351355 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg5qb\" (UniqueName: \"kubernetes.io/projected/4a069916-bc79-42d8-8b30-be79dde85d61-kube-api-access-qg5qb\") pod \"limitador-limitador-7c9b87d574-r5g6w\" (UID: \"4a069916-bc79-42d8-8b30-be79dde85d61\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" Nov 04 04:50:46.408839 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.408815 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" Nov 04 04:50:46.424152 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.424126 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:50:46.776426 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:46.776385 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:50:46.780361 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:50:46.780330 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a069916_bc79_42d8_8b30_be79dde85d61.slice/crio-0cc57dac49ee0fc9b8ba9c390b4c3b3f3308619a40da28c19e5c32e23ac6af7e WatchSource:0}: Error finding container 0cc57dac49ee0fc9b8ba9c390b4c3b3f3308619a40da28c19e5c32e23ac6af7e: Status 404 returned error can't find the container with id 0cc57dac49ee0fc9b8ba9c390b4c3b3f3308619a40da28c19e5c32e23ac6af7e Nov 04 04:50:47.529316 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:47.529284 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" event={"ID":"4a069916-bc79-42d8-8b30-be79dde85d61","Type":"ContainerStarted","Data":"0cc57dac49ee0fc9b8ba9c390b4c3b3f3308619a40da28c19e5c32e23ac6af7e"} Nov 04 04:50:49.270927 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:49.270891 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:49.415515 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:49.415483 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:49.556266 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:49.556189 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:49.861841 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:49.861760 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:50.009783 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:50.009754 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:50.463983 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:50.463946 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:50.606737 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:50.606702 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:50.804574 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:50.804533 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:50.946882 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:50.946825 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:51.545396 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:51.545313 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" event={"ID":"4a069916-bc79-42d8-8b30-be79dde85d61","Type":"ContainerStarted","Data":"feaaf6bc9a0426cced5872951b26f7781837b84221971d74892a206e169e21cc"} Nov 04 04:50:51.545744 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:51.545398 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" Nov 04 04:50:51.654779 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:51.654729 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" podStartSLOduration=1.185123144 podStartE2EDuration="5.654713574s" totalImagesPullingTime="4.46959043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:50:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:50:51.653840767 +0000 UTC m=+293.365531924" watchObservedRunningTime="2026-11-04 04:50:51.654713574 +0000 UTC m=+293.366404712" Nov 04 04:50:55.831739 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:50:55.831705 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:50:57.267623 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:57.267579 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:50:57.409352 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:50:57.409323 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:01.505702 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:01.505656 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:01.649972 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:01.649936 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:02.550243 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:02.550214 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-r5g6w" Nov 04 04:51:02.855709 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:02.855636 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:04.518608 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:04.518577 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mxvsk" Nov 04 04:51:06.061605 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:06.061571 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:06.206557 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:06.206524 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:06.460859 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:06.460780 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:06.603624 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:06.603592 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:13.064100 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:13.064011 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:13.205236 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:13.205198 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:13.463574 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:13.463498 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:13.607223 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:13.607188 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:13.860936 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:13.860892 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:14.007562 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:14.007533 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:16.865214 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:16.865181 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:17.012259 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:17.012227 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:17.160663 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:17.160583 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:17.602107 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:17.602072 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:21.058468 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:21.058434 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:21.200157 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:21.200122 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:24.474115 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:24.474080 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:24.620622 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:24.620574 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:25.838065 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:51:25.838023 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:51:26.260933 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:26.260904 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:26.402974 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:26.402941 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:29.273039 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:29.273002 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:29.419999 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:29.419971 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:38.411127 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:38.411090 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:38.554568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:38.554537 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:38.707807 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:38.707728 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:38.857510 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:38.857476 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:40.129234 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.129198 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk"] Nov 04 04:51:40.136542 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.136516 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.182559 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.182520 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7b15653-232e-4230-b0db-0db35ede9f6f-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.182735 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.182584 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7b15653-232e-4230-b0db-0db35ede9f6f-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.182735 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.182700 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7b15653-232e-4230-b0db-0db35ede9f6f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.182842 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.182758 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7b15653-232e-4230-b0db-0db35ede9f6f-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.182926 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.182833 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7b15653-232e-4230-b0db-0db35ede9f6f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.182984 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.182933 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h84d4\" (UniqueName: \"kubernetes.io/projected/e7b15653-232e-4230-b0db-0db35ede9f6f-kube-api-access-h84d4\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.183037 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.182991 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7b15653-232e-4230-b0db-0db35ede9f6f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.232795 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.232764 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk"] Nov 04 04:51:40.283363 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.283326 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7b15653-232e-4230-b0db-0db35ede9f6f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.283548 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.283375 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7b15653-232e-4230-b0db-0db35ede9f6f-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.283548 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.283399 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7b15653-232e-4230-b0db-0db35ede9f6f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.283548 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.283424 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h84d4\" (UniqueName: \"kubernetes.io/projected/e7b15653-232e-4230-b0db-0db35ede9f6f-kube-api-access-h84d4\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.283548 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.283445 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7b15653-232e-4230-b0db-0db35ede9f6f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.283548 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.283478 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7b15653-232e-4230-b0db-0db35ede9f6f-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.283548 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.283509 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7b15653-232e-4230-b0db-0db35ede9f6f-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.284194 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.284159 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e7b15653-232e-4230-b0db-0db35ede9f6f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.285915 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.285892 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7b15653-232e-4230-b0db-0db35ede9f6f-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.286109 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.286084 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e7b15653-232e-4230-b0db-0db35ede9f6f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.286219 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.286201 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e7b15653-232e-4230-b0db-0db35ede9f6f-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.286309 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.286290 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e7b15653-232e-4230-b0db-0db35ede9f6f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.340490 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.340458 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7b15653-232e-4230-b0db-0db35ede9f6f-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.340765 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.340745 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h84d4\" (UniqueName: \"kubernetes.io/projected/e7b15653-232e-4230-b0db-0db35ede9f6f-kube-api-access-h84d4\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwxsk\" (UID: \"e7b15653-232e-4230-b0db-0db35ede9f6f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.447774 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.447695 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:40.636752 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.636719 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:40.860209 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.860079 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk"] Nov 04 04:51:40.863127 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:51:40.863097 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b15653_232e_4230_b0db_0db35ede9f6f.slice/crio-cac0c1da97c742ed587da120d648261d280f768845a129445d0a4608b343a408 WatchSource:0}: Error finding container cac0c1da97c742ed587da120d648261d280f768845a129445d0a4608b343a408: Status 404 returned error can't find the container with id cac0c1da97c742ed587da120d648261d280f768845a129445d0a4608b343a408 Nov 04 04:51:40.865355 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.865315 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:51:40.865448 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:40.865382 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:51:41.742575 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:41.742538 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" event={"ID":"e7b15653-232e-4230-b0db-0db35ede9f6f","Type":"ContainerStarted","Data":"c83d5bc274f7f65862714eea5f96c7405d2624d77accd43a86aab05db94285f4"} Nov 04 04:51:41.742994 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:41.742595 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:41.742994 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:41.742611 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" event={"ID":"e7b15653-232e-4230-b0db-0db35ede9f6f","Type":"ContainerStarted","Data":"cac0c1da97c742ed587da120d648261d280f768845a129445d0a4608b343a408"} Nov 04 04:51:41.744332 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:41.744308 4374 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-cwxsk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 04:51:41.744454 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:41.744354 4374 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" podUID="e7b15653-232e-4230-b0db-0db35ede9f6f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 04:51:41.935925 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:41.935845 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" podStartSLOduration=1.935830863 podStartE2EDuration="1.935830863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:51:41.901938015 +0000 UTC m=+343.613629154" watchObservedRunningTime="2026-11-04 04:51:41.935830863 +0000 UTC m=+343.647522001" Nov 04 04:51:42.747005 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:42.746978 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwxsk" Nov 04 04:51:42.956836 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:42.956800 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4"] Nov 04 04:51:42.957321 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:42.957084 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" podUID="22b15e41-5032-490c-a660-0bc50e3d4750" containerName="discovery" containerID="cri-o://6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb" gracePeriod=30 Nov 04 04:51:43.183050 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.183018 4374 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-xc7p4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.18:8080/ready\": dial tcp 10.133.0.18:8080: connect: connection refused" start-of-body= Nov 04 04:51:43.183173 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.183093 4374 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" podUID="22b15e41-5032-490c-a660-0bc50e3d4750" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.18:8080/ready\": dial tcp 10.133.0.18:8080: connect: connection refused" Nov 04 04:51:43.299863 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.299836 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:51:43.412488 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.412397 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22b15e41-5032-490c-a660-0bc50e3d4750-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/22b15e41-5032-490c-a660-0bc50e3d4750-local-certs\") pod \"22b15e41-5032-490c-a660-0bc50e3d4750\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " Nov 04 04:51:43.412488 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.412467 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-istio-token\" (UniqueName: \"kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-istio-token\") pod \"22b15e41-5032-490c-a660-0bc50e3d4750\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " Nov 04 04:51:43.412717 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.412568 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-dns-cert\") pod \"22b15e41-5032-490c-a660-0bc50e3d4750\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " Nov 04 04:51:43.412717 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.412605 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-kube-api-access-mmrwf\" (UniqueName: \"kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-kube-api-access-mmrwf\") pod \"22b15e41-5032-490c-a660-0bc50e3d4750\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " Nov 04 04:51:43.412717 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.412642 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-kubeconfig\") pod \"22b15e41-5032-490c-a660-0bc50e3d4750\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " Nov 04 04:51:43.412717 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.412671 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-ca-configmap\") pod \"22b15e41-5032-490c-a660-0bc50e3d4750\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " Nov 04 04:51:43.412717 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.412694 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-cacerts\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-cacerts\") pod \"22b15e41-5032-490c-a660-0bc50e3d4750\" (UID: \"22b15e41-5032-490c-a660-0bc50e3d4750\") " Nov 04 04:51:43.413518 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.413482 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-ca-configmap" pod "22b15e41-5032-490c-a660-0bc50e3d4750" (UID: "22b15e41-5032-490c-a660-0bc50e3d4750"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:51:43.415332 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.415278 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-istio-token" pod "22b15e41-5032-490c-a660-0bc50e3d4750" (UID: "22b15e41-5032-490c-a660-0bc50e3d4750"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:51:43.415462 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.415306 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-cacerts" pod "22b15e41-5032-490c-a660-0bc50e3d4750" (UID: "22b15e41-5032-490c-a660-0bc50e3d4750"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:51:43.415551 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.415529 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22b15e41-5032-490c-a660-0bc50e3d4750-local-certs" pod "22b15e41-5032-490c-a660-0bc50e3d4750" (UID: "22b15e41-5032-490c-a660-0bc50e3d4750"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 04:51:43.415668 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.415649 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-dns-cert" pod "22b15e41-5032-490c-a660-0bc50e3d4750" (UID: "22b15e41-5032-490c-a660-0bc50e3d4750"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:51:43.417178 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.417149 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-kubeconfig" pod "22b15e41-5032-490c-a660-0bc50e3d4750" (UID: "22b15e41-5032-490c-a660-0bc50e3d4750"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:51:43.418052 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.418028 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-kube-api-access-mmrwf" pod "22b15e41-5032-490c-a660-0bc50e3d4750" (UID: "22b15e41-5032-490c-a660-0bc50e3d4750"). InnerVolumeSpecName "kube-api-access-mmrwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:51:43.514200 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.514167 4374 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-dns-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:51:43.514200 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.514195 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmrwf\" (UniqueName: \"kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-kube-api-access-mmrwf\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:51:43.514200 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.514206 4374 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-istio-kubeconfig\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:51:43.514427 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.514216 4374 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/22b15e41-5032-490c-a660-0bc50e3d4750-istio-csr-ca-configmap\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:51:43.514427 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.514224 4374 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/22b15e41-5032-490c-a660-0bc50e3d4750-cacerts\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:51:43.514427 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.514233 4374 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/22b15e41-5032-490c-a660-0bc50e3d4750-local-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:51:43.514427 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.514241 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22b15e41-5032-490c-a660-0bc50e3d4750-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:51:43.650436 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.650402 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:43.751920 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.751861 4374 generic.go:356] "Generic (PLEG): container finished" podID="22b15e41-5032-490c-a660-0bc50e3d4750" containerID="6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb" exitCode=0 Nov 04 04:51:43.751920 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.751919 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" Nov 04 04:51:43.752400 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.751974 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" event={"ID":"22b15e41-5032-490c-a660-0bc50e3d4750","Type":"ContainerDied","Data":"6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb"} Nov 04 04:51:43.752400 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.752025 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4" event={"ID":"22b15e41-5032-490c-a660-0bc50e3d4750","Type":"ContainerDied","Data":"83bdb3c1e91383b305f22e38909f71d3f3681e00a8b9eb120d1dd5dd5f28ffda"} Nov 04 04:51:43.752400 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.752058 4374 scope.go:122] "RemoveContainer" containerID="6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb" Nov 04 04:51:43.762145 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.762107 4374 scope.go:122] "RemoveContainer" containerID="6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb" Nov 04 04:51:43.762460 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:51:43.762441 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb\": container with ID starting with 6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb not found: ID does not exist" containerID="6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb" Nov 04 04:51:43.762546 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.762472 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb"} err="failed to get container status \"6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb\": rpc error: code = NotFound desc = could not find container \"6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb\": container with ID starting with 6ec20e166431bde0678fd88b2f7e09c337871bd6f4e2de4717686bc56dab1edb not found: ID does not exist" Nov 04 04:51:43.790348 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.790326 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:43.907745 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.907715 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4"] Nov 04 04:51:43.917260 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:43.917231 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-xc7p4"] Nov 04 04:51:44.410201 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:44.410169 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22b15e41-5032-490c-a660-0bc50e3d4750" path="/var/lib/kubelet/pods/22b15e41-5032-490c-a660-0bc50e3d4750/volumes" Nov 04 04:51:44.547181 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:44.547146 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:44.688333 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:44.688248 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:44.838475 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:44.838436 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:45.062299 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:45.062261 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:45.205073 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:45.205042 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:45.349691 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:45.349595 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:45.526145 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:45.526112 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:45.672909 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:45.672797 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:46.703624 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:46.703592 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:47.254589 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:47.254558 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:47.431055 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:47.431023 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:51.411911 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:51.411862 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:51.555443 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:51.555408 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:51.698224 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:51.698134 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:51.842806 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:51.842775 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:51.983967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:51.983938 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:52.126846 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:52.126796 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:52.267406 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:52.267334 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:52.413645 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:52.413617 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:52.555346 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:51:52.555272 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:51:55.831560 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:51:55.831513 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:52:00.888040 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:00.888006 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:17.398246 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:17.398211 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:17.539995 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:17.539961 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:19.519882 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:19.519845 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:19.683939 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:19.683902 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:19.829084 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:19.828998 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:19.969195 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:19.969163 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:20.562267 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.562233 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4"] Nov 04 04:52:20.562620 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.562550 4374 cpu_manager.go:414] "containerMap: removing container" podUID="22b15e41-5032-490c-a660-0bc50e3d4750" containerName="discovery" Nov 04 04:52:20.562620 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.562560 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22b15e41-5032-490c-a660-0bc50e3d4750" containerName="discovery" Nov 04 04:52:20.562620 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.562613 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="22b15e41-5032-490c-a660-0bc50e3d4750" containerName="discovery" Nov 04 04:52:20.565560 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.565541 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.622033 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.622005 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cc424a1-07f1-413c-b972-4403da62c53c-openshift-service-ca\") pod \"maas-api-674d648f7b-wg4g4\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.622154 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.622062 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzqw2\" (UniqueName: \"kubernetes.io/projected/2cc424a1-07f1-413c-b972-4403da62c53c-kube-api-access-wzqw2\") pod \"maas-api-674d648f7b-wg4g4\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.622196 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.622155 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-metrics-certs\") pod \"maas-api-674d648f7b-wg4g4\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.622196 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.622191 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-maas-api-tls\") pod \"maas-api-674d648f7b-wg4g4\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.661114 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.661083 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 04:52:20.661114 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.661104 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bv7vs\"" Nov 04 04:52:20.661296 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.661133 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Nov 04 04:52:20.661296 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.661152 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 04 04:52:20.661296 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.661211 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 04:52:20.662575 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.662557 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4"] Nov 04 04:52:20.722789 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.722761 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzqw2\" (UniqueName: \"kubernetes.io/projected/2cc424a1-07f1-413c-b972-4403da62c53c-kube-api-access-wzqw2\") pod \"maas-api-674d648f7b-wg4g4\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.722989 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.722804 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-metrics-certs\") pod \"maas-api-674d648f7b-wg4g4\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.722989 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.722825 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-maas-api-tls\") pod \"maas-api-674d648f7b-wg4g4\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.722989 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.722862 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cc424a1-07f1-413c-b972-4403da62c53c-openshift-service-ca\") pod \"maas-api-674d648f7b-wg4g4\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.723532 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.723511 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cc424a1-07f1-413c-b972-4403da62c53c-openshift-service-ca\") pod \"maas-api-674d648f7b-wg4g4\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.725499 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.725469 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-metrics-certs\") pod \"maas-api-674d648f7b-wg4g4\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.725610 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.725477 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-maas-api-tls\") pod \"maas-api-674d648f7b-wg4g4\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.779730 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.779702 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzqw2\" (UniqueName: \"kubernetes.io/projected/2cc424a1-07f1-413c-b972-4403da62c53c-kube-api-access-wzqw2\") pod \"maas-api-674d648f7b-wg4g4\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:20.878034 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:20.877959 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:21.243710 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:21.243679 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4"] Nov 04 04:52:21.246766 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:52:21.246733 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cc424a1_07f1_413c_b972_4403da62c53c.slice/crio-064badc8b9a2b1662de3f2f8d2716e93b4b1d43b916710e9596a0f607a1e7b1c WatchSource:0}: Error finding container 064badc8b9a2b1662de3f2f8d2716e93b4b1d43b916710e9596a0f607a1e7b1c: Status 404 returned error can't find the container with id 064badc8b9a2b1662de3f2f8d2716e93b4b1d43b916710e9596a0f607a1e7b1c Nov 04 04:52:21.248086 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:21.248067 4374 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 04:52:21.900224 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:21.900185 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" event={"ID":"2cc424a1-07f1-413c-b972-4403da62c53c","Type":"ContainerStarted","Data":"064badc8b9a2b1662de3f2f8d2716e93b4b1d43b916710e9596a0f607a1e7b1c"} Nov 04 04:52:23.029407 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:23.029369 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:23.170437 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:23.170405 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:23.315615 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:23.315534 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:25.502036 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:25.501998 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:52:25.832302 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:52:25.832209 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:52:27.925768 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:27.925734 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" event={"ID":"2cc424a1-07f1-413c-b972-4403da62c53c","Type":"ContainerStarted","Data":"db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c"} Nov 04 04:52:27.926166 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:27.925815 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:28.036556 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:28.036500 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" podStartSLOduration=2.412440971 podStartE2EDuration="8.036484453s" totalImagesPullingTime="5.624043482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:52:28.035005902 +0000 UTC m=+389.746697067" watchObservedRunningTime="2026-11-04 04:52:28.036484453 +0000 UTC m=+389.748175592" Nov 04 04:52:33.934042 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:33.934016 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:34.449641 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:34.449606 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:34.651112 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:34.651082 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:34.929168 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:34.929137 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-597b465c59-jhv84"] Nov 04 04:52:34.932594 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:34.932573 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.029065 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.029035 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-597b465c59-jhv84"] Nov 04 04:52:35.037718 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.037697 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72c4479b-c93d-40d3-b479-3ea6c557515d-metrics-certs\") pod \"maas-api-597b465c59-jhv84\" (UID: \"72c4479b-c93d-40d3-b479-3ea6c557515d\") " pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.037815 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.037727 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsnl7\" (UniqueName: \"kubernetes.io/projected/72c4479b-c93d-40d3-b479-3ea6c557515d-kube-api-access-jsnl7\") pod \"maas-api-597b465c59-jhv84\" (UID: \"72c4479b-c93d-40d3-b479-3ea6c557515d\") " pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.037893 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.037855 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72c4479b-c93d-40d3-b479-3ea6c557515d-maas-api-tls\") pod \"maas-api-597b465c59-jhv84\" (UID: \"72c4479b-c93d-40d3-b479-3ea6c557515d\") " pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.037965 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.037947 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72c4479b-c93d-40d3-b479-3ea6c557515d-openshift-service-ca\") pod \"maas-api-597b465c59-jhv84\" (UID: \"72c4479b-c93d-40d3-b479-3ea6c557515d\") " pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.139193 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.139156 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72c4479b-c93d-40d3-b479-3ea6c557515d-openshift-service-ca\") pod \"maas-api-597b465c59-jhv84\" (UID: \"72c4479b-c93d-40d3-b479-3ea6c557515d\") " pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.139352 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.139228 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72c4479b-c93d-40d3-b479-3ea6c557515d-metrics-certs\") pod \"maas-api-597b465c59-jhv84\" (UID: \"72c4479b-c93d-40d3-b479-3ea6c557515d\") " pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.139352 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.139246 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsnl7\" (UniqueName: \"kubernetes.io/projected/72c4479b-c93d-40d3-b479-3ea6c557515d-kube-api-access-jsnl7\") pod \"maas-api-597b465c59-jhv84\" (UID: \"72c4479b-c93d-40d3-b479-3ea6c557515d\") " pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.139352 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.139294 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72c4479b-c93d-40d3-b479-3ea6c557515d-maas-api-tls\") pod \"maas-api-597b465c59-jhv84\" (UID: \"72c4479b-c93d-40d3-b479-3ea6c557515d\") " pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.139947 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.139918 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72c4479b-c93d-40d3-b479-3ea6c557515d-openshift-service-ca\") pod \"maas-api-597b465c59-jhv84\" (UID: \"72c4479b-c93d-40d3-b479-3ea6c557515d\") " pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.141743 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.141719 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72c4479b-c93d-40d3-b479-3ea6c557515d-metrics-certs\") pod \"maas-api-597b465c59-jhv84\" (UID: \"72c4479b-c93d-40d3-b479-3ea6c557515d\") " pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.141839 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.141767 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72c4479b-c93d-40d3-b479-3ea6c557515d-maas-api-tls\") pod \"maas-api-597b465c59-jhv84\" (UID: \"72c4479b-c93d-40d3-b479-3ea6c557515d\") " pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.199704 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.199642 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsnl7\" (UniqueName: \"kubernetes.io/projected/72c4479b-c93d-40d3-b479-3ea6c557515d-kube-api-access-jsnl7\") pod \"maas-api-597b465c59-jhv84\" (UID: \"72c4479b-c93d-40d3-b479-3ea6c557515d\") " pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.244733 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.244698 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:35.610643 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.610609 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-597b465c59-jhv84"] Nov 04 04:52:35.613751 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:52:35.613717 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72c4479b_c93d_40d3_b479_3ea6c557515d.slice/crio-1e5eb5d08976c2d5f3eae1ccb8ff57149bd021432098dbbfadd329a5d09296bf WatchSource:0}: Error finding container 1e5eb5d08976c2d5f3eae1ccb8ff57149bd021432098dbbfadd329a5d09296bf: Status 404 returned error can't find the container with id 1e5eb5d08976c2d5f3eae1ccb8ff57149bd021432098dbbfadd329a5d09296bf Nov 04 04:52:35.958430 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:35.958347 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" event={"ID":"72c4479b-c93d-40d3-b479-3ea6c557515d","Type":"ContainerStarted","Data":"1e5eb5d08976c2d5f3eae1ccb8ff57149bd021432098dbbfadd329a5d09296bf"} Nov 04 04:52:36.963458 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:36.963378 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" event={"ID":"72c4479b-c93d-40d3-b479-3ea6c557515d","Type":"ContainerStarted","Data":"4b38bc01306056cdbfe2cdc376268427096e018fbe5d6f075c527c8ac75a7414"} Nov 04 04:52:36.963830 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:36.963499 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:37.120736 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:37.120674 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" podStartSLOduration=2.145917571 podStartE2EDuration="3.120660581s" totalImagesPullingTime="974.74301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:52:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:52:37.119669128 +0000 UTC m=+398.831360269" watchObservedRunningTime="2026-11-04 04:52:37.120660581 +0000 UTC m=+398.832351720" Nov 04 04:52:41.001477 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:41.001392 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:42.972684 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:42.972657 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-597b465c59-jhv84" Nov 04 04:52:43.131625 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.131595 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4"] Nov 04 04:52:43.131856 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.131814 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" podUID="2cc424a1-07f1-413c-b972-4403da62c53c" containerName="maas-api" containerID="cri-o://db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c" gracePeriod=30 Nov 04 04:52:43.372079 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.372056 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:43.507255 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.507180 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2cc424a1-07f1-413c-b972-4403da62c53c-kube-api-access-wzqw2\" (UniqueName: \"kubernetes.io/projected/2cc424a1-07f1-413c-b972-4403da62c53c-kube-api-access-wzqw2\") pod \"2cc424a1-07f1-413c-b972-4403da62c53c\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " Nov 04 04:52:43.507255 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.507216 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-metrics-certs\") pod \"2cc424a1-07f1-413c-b972-4403da62c53c\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " Nov 04 04:52:43.507255 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.507252 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-maas-api-tls\") pod \"2cc424a1-07f1-413c-b972-4403da62c53c\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " Nov 04 04:52:43.507537 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.507295 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2cc424a1-07f1-413c-b972-4403da62c53c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cc424a1-07f1-413c-b972-4403da62c53c-openshift-service-ca\") pod \"2cc424a1-07f1-413c-b972-4403da62c53c\" (UID: \"2cc424a1-07f1-413c-b972-4403da62c53c\") " Nov 04 04:52:43.507686 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.507661 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cc424a1-07f1-413c-b972-4403da62c53c-openshift-service-ca" pod "2cc424a1-07f1-413c-b972-4403da62c53c" (UID: "2cc424a1-07f1-413c-b972-4403da62c53c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 04:52:43.509409 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.509385 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-maas-api-tls" pod "2cc424a1-07f1-413c-b972-4403da62c53c" (UID: "2cc424a1-07f1-413c-b972-4403da62c53c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:52:43.509509 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.509432 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-metrics-certs" pod "2cc424a1-07f1-413c-b972-4403da62c53c" (UID: "2cc424a1-07f1-413c-b972-4403da62c53c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:52:43.509509 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.509444 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cc424a1-07f1-413c-b972-4403da62c53c-kube-api-access-wzqw2" pod "2cc424a1-07f1-413c-b972-4403da62c53c" (UID: "2cc424a1-07f1-413c-b972-4403da62c53c"). InnerVolumeSpecName "kube-api-access-wzqw2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:52:43.608740 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.608709 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzqw2\" (UniqueName: \"kubernetes.io/projected/2cc424a1-07f1-413c-b972-4403da62c53c-kube-api-access-wzqw2\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:52:43.608740 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.608736 4374 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-metrics-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:52:43.608740 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.608748 4374 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cc424a1-07f1-413c-b972-4403da62c53c-maas-api-tls\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:52:43.608960 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.608758 4374 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cc424a1-07f1-413c-b972-4403da62c53c-openshift-service-ca\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:52:43.992305 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.992275 4374 generic.go:356] "Generic (PLEG): container finished" podID="2cc424a1-07f1-413c-b972-4403da62c53c" containerID="db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c" exitCode=0 Nov 04 04:52:43.992699 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.992317 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" Nov 04 04:52:43.992699 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.992321 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" event={"ID":"2cc424a1-07f1-413c-b972-4403da62c53c","Type":"ContainerDied","Data":"db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c"} Nov 04 04:52:43.992699 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.992428 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4" event={"ID":"2cc424a1-07f1-413c-b972-4403da62c53c","Type":"ContainerDied","Data":"064badc8b9a2b1662de3f2f8d2716e93b4b1d43b916710e9596a0f607a1e7b1c"} Nov 04 04:52:43.992699 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:43.992450 4374 scope.go:122] "RemoveContainer" containerID="db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c" Nov 04 04:52:44.001095 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:44.001076 4374 scope.go:122] "RemoveContainer" containerID="db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c" Nov 04 04:52:44.001354 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:52:44.001334 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c\": container with ID starting with db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c not found: ID does not exist" containerID="db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c" Nov 04 04:52:44.001408 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:44.001365 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c"} err="failed to get container status \"db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c\": rpc error: code = NotFound desc = could not find container \"db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c\": container with ID starting with db47979ae26f836b69e323dcccb1b1a871ad2e0a3773718aaf5a548ca33e5c0c not found: ID does not exist" Nov 04 04:52:44.146015 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:44.145980 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4"] Nov 04 04:52:44.192380 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:44.192352 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-674d648f7b-wg4g4"] Nov 04 04:52:44.410136 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:44.410060 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2cc424a1-07f1-413c-b972-4403da62c53c" path="/var/lib/kubelet/pods/2cc424a1-07f1-413c-b972-4403da62c53c/volumes" Nov 04 04:52:45.622177 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:45.622126 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:45.766033 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:52:45.766001 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:52:55.832065 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:52:55.832030 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:53:25.831962 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:53:25.831923 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:53:35.853349 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:35.853316 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:53:35.998202 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:35.998170 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:53:41.336746 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:41.336710 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:53:54.391970 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.391936 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69"] Nov 04 04:53:54.392351 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.392251 4374 cpu_manager.go:414] "containerMap: removing container" podUID="2cc424a1-07f1-413c-b972-4403da62c53c" containerName="maas-api" Nov 04 04:53:54.392351 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.392261 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2cc424a1-07f1-413c-b972-4403da62c53c" containerName="maas-api" Nov 04 04:53:54.392351 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.392309 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="2cc424a1-07f1-413c-b972-4403da62c53c" containerName="maas-api" Nov 04 04:53:54.395292 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.395273 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.444299 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.444267 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Nov 04 04:53:54.467305 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.467274 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Nov 04 04:53:54.467305 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.467288 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Nov 04 04:53:54.490388 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.490359 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-67mz4\"" Nov 04 04:53:54.490571 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.490550 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6befc7e-171a-468b-a6b1-55c469a0cd6e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.490613 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.490602 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvhr9\" (UniqueName: \"kubernetes.io/projected/d6befc7e-171a-468b-a6b1-55c469a0cd6e-kube-api-access-wvhr9\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.490748 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.490730 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6befc7e-171a-468b-a6b1-55c469a0cd6e-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.490822 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.490762 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6befc7e-171a-468b-a6b1-55c469a0cd6e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.490905 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.490841 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6befc7e-171a-468b-a6b1-55c469a0cd6e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.490968 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.490925 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6befc7e-171a-468b-a6b1-55c469a0cd6e-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.492315 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.492281 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:53:54.492692 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.492668 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69"] Nov 04 04:53:54.591742 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.591709 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvhr9\" (UniqueName: \"kubernetes.io/projected/d6befc7e-171a-468b-a6b1-55c469a0cd6e-kube-api-access-wvhr9\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.591915 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.591753 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6befc7e-171a-468b-a6b1-55c469a0cd6e-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.591915 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.591770 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6befc7e-171a-468b-a6b1-55c469a0cd6e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.592038 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.591919 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6befc7e-171a-468b-a6b1-55c469a0cd6e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.592038 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.591970 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6befc7e-171a-468b-a6b1-55c469a0cd6e-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.592172 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.592043 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6befc7e-171a-468b-a6b1-55c469a0cd6e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.592172 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.592120 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6befc7e-171a-468b-a6b1-55c469a0cd6e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.592236 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.592187 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6befc7e-171a-468b-a6b1-55c469a0cd6e-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.592320 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.592304 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6befc7e-171a-468b-a6b1-55c469a0cd6e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.594158 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.594140 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6befc7e-171a-468b-a6b1-55c469a0cd6e-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.594339 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.594323 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6befc7e-171a-468b-a6b1-55c469a0cd6e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.651561 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.651498 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvhr9\" (UniqueName: \"kubernetes.io/projected/d6befc7e-171a-468b-a6b1-55c469a0cd6e-kube-api-access-wvhr9\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-rxz69\" (UID: \"d6befc7e-171a-468b-a6b1-55c469a0cd6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:54.707137 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:54.707110 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:55.069559 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:55.069536 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69"] Nov 04 04:53:55.072191 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:53:55.072156 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6befc7e_171a_468b_a6b1_55c469a0cd6e.slice/crio-dd17d6de2efc20157b83cb24f23ecdc5d0e8c86c7b369ea28324d84e4866494d WatchSource:0}: Error finding container dd17d6de2efc20157b83cb24f23ecdc5d0e8c86c7b369ea28324d84e4866494d: Status 404 returned error can't find the container with id dd17d6de2efc20157b83cb24f23ecdc5d0e8c86c7b369ea28324d84e4866494d Nov 04 04:53:55.270582 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:55.270549 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" event={"ID":"d6befc7e-171a-468b-a6b1-55c469a0cd6e","Type":"ContainerStarted","Data":"dd17d6de2efc20157b83cb24f23ecdc5d0e8c86c7b369ea28324d84e4866494d"} Nov 04 04:53:55.831123 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:53:55.831092 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:53:57.280826 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.280791 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" event={"ID":"d6befc7e-171a-468b-a6b1-55c469a0cd6e","Type":"ContainerStarted","Data":"0fcef54c3b7b44652145adaa92ef62bc020567a5cbca4f1b98956862d5a8cf4a"} Nov 04 04:53:57.391839 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.391791 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" podStartSLOduration=1.5050621629999998 podStartE2EDuration="3.391777567s" totalImagesPullingTime="1.886715404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:53:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:53:57.390537714 +0000 UTC m=+479.102228852" watchObservedRunningTime="2026-11-04 04:53:57.391777567 +0000 UTC m=+479.103468710" Nov 04 04:53:57.637542 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.637460 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2"] Nov 04 04:53:57.640917 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.640895 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.689738 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.689696 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Nov 04 04:53:57.695851 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.695822 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2"] Nov 04 04:53:57.707285 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.707262 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:53:57.735524 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.735490 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:53:57.759574 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.759548 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:53:57.820459 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.820402 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4903fa82-ba60-4ad8-9a97-622f6e6361f0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.820621 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.820534 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4903fa82-ba60-4ad8-9a97-622f6e6361f0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.820664 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.820619 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4903fa82-ba60-4ad8-9a97-622f6e6361f0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.820664 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.820642 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4903fa82-ba60-4ad8-9a97-622f6e6361f0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.820664 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.820659 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4903fa82-ba60-4ad8-9a97-622f6e6361f0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.820764 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.820684 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqn4b\" (UniqueName: \"kubernetes.io/projected/4903fa82-ba60-4ad8-9a97-622f6e6361f0-kube-api-access-jqn4b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.921898 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.921794 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4903fa82-ba60-4ad8-9a97-622f6e6361f0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.921898 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.921844 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4903fa82-ba60-4ad8-9a97-622f6e6361f0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.921898 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.921883 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4903fa82-ba60-4ad8-9a97-622f6e6361f0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.921898 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.921901 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4903fa82-ba60-4ad8-9a97-622f6e6361f0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.922287 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.921941 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqn4b\" (UniqueName: \"kubernetes.io/projected/4903fa82-ba60-4ad8-9a97-622f6e6361f0-kube-api-access-jqn4b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.922287 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.922007 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4903fa82-ba60-4ad8-9a97-622f6e6361f0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.922287 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.922271 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4903fa82-ba60-4ad8-9a97-622f6e6361f0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.922450 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.922341 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4903fa82-ba60-4ad8-9a97-622f6e6361f0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.922545 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.922520 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4903fa82-ba60-4ad8-9a97-622f6e6361f0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.924581 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.924557 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4903fa82-ba60-4ad8-9a97-622f6e6361f0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.924701 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.924588 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4903fa82-ba60-4ad8-9a97-622f6e6361f0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:57.979545 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:57.979509 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqn4b\" (UniqueName: \"kubernetes.io/projected/4903fa82-ba60-4ad8-9a97-622f6e6361f0-kube-api-access-jqn4b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2\" (UID: \"4903fa82-ba60-4ad8-9a97-622f6e6361f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:58.253658 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:58.253625 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:53:58.618900 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:58.618806 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2"] Nov 04 04:53:58.622539 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:53:58.622497 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4903fa82_ba60_4ad8_9a97_622f6e6361f0.slice/crio-60bd057550531f0c5bddb6dcbdb7c1cd04cfc773abb9517c696003882b652250 WatchSource:0}: Error finding container 60bd057550531f0c5bddb6dcbdb7c1cd04cfc773abb9517c696003882b652250: Status 404 returned error can't find the container with id 60bd057550531f0c5bddb6dcbdb7c1cd04cfc773abb9517c696003882b652250 Nov 04 04:53:59.291540 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:59.291499 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" event={"ID":"4903fa82-ba60-4ad8-9a97-622f6e6361f0","Type":"ContainerStarted","Data":"5d063fcbf20ab790a72a85fbb6101087320c1a77b29468c316a21b7ba7adcd7e"} Nov 04 04:53:59.291626 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:59.291578 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" event={"ID":"4903fa82-ba60-4ad8-9a97-622f6e6361f0","Type":"ContainerStarted","Data":"60bd057550531f0c5bddb6dcbdb7c1cd04cfc773abb9517c696003882b652250"} Nov 04 04:53:59.401306 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:53:59.401206 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" podStartSLOduration=1.79662648 podStartE2EDuration="2.40118686s" totalImagesPullingTime="604.56038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:53:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:53:59.400770286 +0000 UTC m=+481.112461425" watchObservedRunningTime="2026-11-04 04:53:59.40118686 +0000 UTC m=+481.112877999" Nov 04 04:54:00.254687 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:00.254652 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:54:02.645484 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:02.645455 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:03.745149 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:03.745115 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:03.886573 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:03.886546 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:04.707376 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:04.707341 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:54:04.720031 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:04.720006 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:54:05.326052 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:05.326026 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-rxz69" Nov 04 04:54:05.821499 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:05.821465 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:54:06.467477 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.467446 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh"] Nov 04 04:54:06.470847 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.470827 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.494615 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.494586 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbvn2\" (UniqueName: \"kubernetes.io/projected/b023d5dd-1b1e-4c6e-8124-7fc49137d601-kube-api-access-tbvn2\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.494708 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.494621 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b023d5dd-1b1e-4c6e-8124-7fc49137d601-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.494708 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.494640 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b023d5dd-1b1e-4c6e-8124-7fc49137d601-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.494775 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.494757 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b023d5dd-1b1e-4c6e-8124-7fc49137d601-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.494809 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.494799 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b023d5dd-1b1e-4c6e-8124-7fc49137d601-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.494938 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.494924 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b023d5dd-1b1e-4c6e-8124-7fc49137d601-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.519998 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.519975 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Nov 04 04:54:06.560676 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.560648 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:06.567473 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.567451 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh"] Nov 04 04:54:06.595385 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.595356 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b023d5dd-1b1e-4c6e-8124-7fc49137d601-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.595500 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.595399 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b023d5dd-1b1e-4c6e-8124-7fc49137d601-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.595500 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.595442 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b023d5dd-1b1e-4c6e-8124-7fc49137d601-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.595500 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.595494 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbvn2\" (UniqueName: \"kubernetes.io/projected/b023d5dd-1b1e-4c6e-8124-7fc49137d601-kube-api-access-tbvn2\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.595661 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.595528 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b023d5dd-1b1e-4c6e-8124-7fc49137d601-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.595661 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.595551 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b023d5dd-1b1e-4c6e-8124-7fc49137d601-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.595899 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.595856 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b023d5dd-1b1e-4c6e-8124-7fc49137d601-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.595970 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.595945 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b023d5dd-1b1e-4c6e-8124-7fc49137d601-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.596005 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.595973 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b023d5dd-1b1e-4c6e-8124-7fc49137d601-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.597609 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.597593 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b023d5dd-1b1e-4c6e-8124-7fc49137d601-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.597911 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.597893 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b023d5dd-1b1e-4c6e-8124-7fc49137d601-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.653356 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.653328 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbvn2\" (UniqueName: \"kubernetes.io/projected/b023d5dd-1b1e-4c6e-8124-7fc49137d601-kube-api-access-tbvn2\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh\" (UID: \"b023d5dd-1b1e-4c6e-8124-7fc49137d601\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:06.782885 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:06.782839 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:07.143814 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:07.143741 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh"] Nov 04 04:54:07.147047 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:54:07.147023 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb023d5dd_1b1e_4c6e_8124_7fc49137d601.slice/crio-11ed7c474c7e9e620e727fcd6f2fb81169055e96fdaf35f174fb1e97e6af6ca0 WatchSource:0}: Error finding container 11ed7c474c7e9e620e727fcd6f2fb81169055e96fdaf35f174fb1e97e6af6ca0: Status 404 returned error can't find the container with id 11ed7c474c7e9e620e727fcd6f2fb81169055e96fdaf35f174fb1e97e6af6ca0 Nov 04 04:54:07.323439 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:07.323403 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" event={"ID":"b023d5dd-1b1e-4c6e-8124-7fc49137d601","Type":"ContainerStarted","Data":"11ed7c474c7e9e620e727fcd6f2fb81169055e96fdaf35f174fb1e97e6af6ca0"} Nov 04 04:54:08.254404 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:08.254363 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:54:08.266621 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:08.266600 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:54:08.330477 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:08.330436 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" event={"ID":"b023d5dd-1b1e-4c6e-8124-7fc49137d601","Type":"ContainerStarted","Data":"61939877ad6e26a504f271e8ff844ccf3b66e57797aa3a400b8c549f62b7ae59"} Nov 04 04:54:08.341395 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:08.341374 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2" Nov 04 04:54:08.596040 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:08.595925 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" podStartSLOduration=1.932535946 podStartE2EDuration="2.595908974s" totalImagesPullingTime="663.373028ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:54:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:54:08.595672333 +0000 UTC m=+490.307363484" watchObservedRunningTime="2026-11-04 04:54:08.595908974 +0000 UTC m=+490.307600116" Nov 04 04:54:08.783319 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:08.783289 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:09.898135 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:09.898101 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw"] Nov 04 04:54:09.901254 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:09.901234 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:09.923831 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:09.923802 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3cd901-b98c-4232-8604-87ef595b35e2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:09.923989 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:09.923842 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3cd901-b98c-4232-8604-87ef595b35e2-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:09.923989 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:09.923894 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3cd901-b98c-4232-8604-87ef595b35e2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:09.923989 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:09.923923 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df8fx\" (UniqueName: \"kubernetes.io/projected/9f3cd901-b98c-4232-8604-87ef595b35e2-kube-api-access-df8fx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:09.924131 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:09.924046 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3cd901-b98c-4232-8604-87ef595b35e2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:09.924131 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:09.924087 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3cd901-b98c-4232-8604-87ef595b35e2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:09.973439 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:09.973410 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Nov 04 04:54:09.993297 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:09.993272 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:10.002650 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:09.999233 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw"] Nov 04 04:54:10.025117 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.025084 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3cd901-b98c-4232-8604-87ef595b35e2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.025117 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.025118 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3cd901-b98c-4232-8604-87ef595b35e2-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.025304 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.025138 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3cd901-b98c-4232-8604-87ef595b35e2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.025304 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.025162 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df8fx\" (UniqueName: \"kubernetes.io/projected/9f3cd901-b98c-4232-8604-87ef595b35e2-kube-api-access-df8fx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.025304 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.025226 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3cd901-b98c-4232-8604-87ef595b35e2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.025304 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.025275 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3cd901-b98c-4232-8604-87ef595b35e2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.025599 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.025574 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3cd901-b98c-4232-8604-87ef595b35e2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.025663 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.025599 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3cd901-b98c-4232-8604-87ef595b35e2-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.025663 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.025659 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3cd901-b98c-4232-8604-87ef595b35e2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.027459 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.027434 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3cd901-b98c-4232-8604-87ef595b35e2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.027713 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.027695 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3cd901-b98c-4232-8604-87ef595b35e2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.082783 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.082745 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-df8fx\" (UniqueName: \"kubernetes.io/projected/9f3cd901-b98c-4232-8604-87ef595b35e2-kube-api-access-df8fx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw\" (UID: \"9f3cd901-b98c-4232-8604-87ef595b35e2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.222662 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.222632 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:10.586179 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:10.586100 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw"] Nov 04 04:54:10.592016 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:54:10.589498 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f3cd901_b98c_4232_8604_87ef595b35e2.slice/crio-fd31818ffb9db8a8435da5f88a7676c7c8348fbcefaa499a35c051a63c62267b WatchSource:0}: Error finding container fd31818ffb9db8a8435da5f88a7676c7c8348fbcefaa499a35c051a63c62267b: Status 404 returned error can't find the container with id fd31818ffb9db8a8435da5f88a7676c7c8348fbcefaa499a35c051a63c62267b Nov 04 04:54:11.344635 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:11.344602 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" event={"ID":"9f3cd901-b98c-4232-8604-87ef595b35e2","Type":"ContainerStarted","Data":"30fe1bc8db2154d8ba88d98cfe5bb821593bdd5b8bb864b5ffbc2fbb5bd8d45b"} Nov 04 04:54:11.344999 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:11.344642 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" event={"ID":"9f3cd901-b98c-4232-8604-87ef595b35e2","Type":"ContainerStarted","Data":"fd31818ffb9db8a8435da5f88a7676c7c8348fbcefaa499a35c051a63c62267b"} Nov 04 04:54:11.456458 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:11.456399 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" podStartSLOduration=1.835406699 podStartE2EDuration="2.45638577s" totalImagesPullingTime="620.979071ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:54:11.455950366 +0000 UTC m=+493.167641519" watchObservedRunningTime="2026-11-04 04:54:11.45638577 +0000 UTC m=+493.168076908" Nov 04 04:54:12.223706 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:12.223671 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:13.130407 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:13.130373 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:15.793350 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:15.793299 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:54:16.783530 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:16.783498 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:16.796065 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:16.796037 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:17.378920 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:17.378890 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh" Nov 04 04:54:18.187331 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:18.187300 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:20.223492 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:20.223459 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:20.235767 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:20.235744 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:20.391067 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:20.391044 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw" Nov 04 04:54:21.024271 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.024235 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w"] Nov 04 04:54:21.027975 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.027956 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.077197 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.077172 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-7dlcv\"" Nov 04 04:54:21.118936 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.118910 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.119054 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.118940 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j4cm\" (UniqueName: \"kubernetes.io/projected/dc3bdc4d-9167-4b54-94a5-a364b1deb668-kube-api-access-6j4cm\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.119054 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.118962 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.119133 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.119102 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.119133 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.119126 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.119201 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.119151 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.119201 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.119173 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.119264 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.119257 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.119299 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.119292 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.125936 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.125914 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w"] Nov 04 04:54:21.220302 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.220267 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.220467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.220310 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.220467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.220354 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.220467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.220377 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j4cm\" (UniqueName: \"kubernetes.io/projected/dc3bdc4d-9167-4b54-94a5-a364b1deb668-kube-api-access-6j4cm\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.220467 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.220405 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.220666 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.220479 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.220666 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.220501 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.220666 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.220535 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.220666 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.220569 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.220980 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.220954 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.221085 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.220959 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.221085 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.221035 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.221085 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.221062 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.221253 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.221232 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.222995 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.222963 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.223196 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.223176 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.278025 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.277951 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc3bdc4d-9167-4b54-94a5-a364b1deb668-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.278453 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.278253 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j4cm\" (UniqueName: \"kubernetes.io/projected/dc3bdc4d-9167-4b54-94a5-a364b1deb668-kube-api-access-6j4cm\") pod \"maas-default-gateway-openshift-default-57f78bcd55-psx2w\" (UID: \"dc3bdc4d-9167-4b54-94a5-a364b1deb668\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.339819 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.339789 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:21.799531 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:21.799497 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w"] Nov 04 04:54:21.803607 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:54:21.803572 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc3bdc4d_9167_4b54_94a5_a364b1deb668.slice/crio-59e63ebdf4ef1d730f5c483f024d8118e458db8d9c7b828ae68fff2808420fb2 WatchSource:0}: Error finding container 59e63ebdf4ef1d730f5c483f024d8118e458db8d9c7b828ae68fff2808420fb2: Status 404 returned error can't find the container with id 59e63ebdf4ef1d730f5c483f024d8118e458db8d9c7b828ae68fff2808420fb2 Nov 04 04:54:22.298504 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.298463 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8"] Nov 04 04:54:22.303359 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.303335 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.352464 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.352429 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Nov 04 04:54:22.389924 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.389894 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" event={"ID":"dc3bdc4d-9167-4b54-94a5-a364b1deb668","Type":"ContainerStarted","Data":"59e63ebdf4ef1d730f5c483f024d8118e458db8d9c7b828ae68fff2808420fb2"} Nov 04 04:54:22.400407 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.400384 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8"] Nov 04 04:54:22.430861 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.430828 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.431006 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.430859 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.431059 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.430997 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.431098 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.431061 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.431098 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.431083 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.431184 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.431114 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z2gj\" (UniqueName: \"kubernetes.io/projected/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-kube-api-access-8z2gj\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.431485 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.431468 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:22.531858 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.531823 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.531858 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.531859 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.532104 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.531924 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.532104 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.531958 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.532104 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.531973 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.532104 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.531994 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z2gj\" (UniqueName: \"kubernetes.io/projected/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-kube-api-access-8z2gj\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.532349 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.532325 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.532413 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.532368 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.532461 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.532427 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.534174 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.534154 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.534360 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.534343 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.589567 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.589510 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z2gj\" (UniqueName: \"kubernetes.io/projected/1b80b52d-3ca9-4459-8ee7-90ce72f98bcf-kube-api-access-8z2gj\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-8txv8\" (UID: \"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.616341 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.616316 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:22.980777 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:22.980700 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8"] Nov 04 04:54:22.983475 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:54:22.983447 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b80b52d_3ca9_4459_8ee7_90ce72f98bcf.slice/crio-a2b424d7aa24377f1b186680e8a6f5a922c0c9d7c73137646b483d06c5954b6d WatchSource:0}: Error finding container a2b424d7aa24377f1b186680e8a6f5a922c0c9d7c73137646b483d06c5954b6d: Status 404 returned error can't find the container with id a2b424d7aa24377f1b186680e8a6f5a922c0c9d7c73137646b483d06c5954b6d Nov 04 04:54:23.396235 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:23.396163 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" event={"ID":"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf","Type":"ContainerStarted","Data":"a2b424d7aa24377f1b186680e8a6f5a922c0c9d7c73137646b483d06c5954b6d"} Nov 04 04:54:24.402447 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:24.402408 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" event={"ID":"1b80b52d-3ca9-4459-8ee7-90ce72f98bcf","Type":"ContainerStarted","Data":"143ba7013581851b077abbb90bd71e33a67e715328a0190b05d5af4b09878da4"} Nov 04 04:54:24.514184 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:24.514126 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" podStartSLOduration=1.914275694 podStartE2EDuration="2.514110734s" totalImagesPullingTime="599.83504ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:54:24.512684089 +0000 UTC m=+506.224375241" watchObservedRunningTime="2026-11-04 04:54:24.514110734 +0000 UTC m=+506.225801873" Nov 04 04:54:24.617099 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:24.617065 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:25.838296 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:54:25.838257 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:54:26.663193 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:26.663155 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:54:26.663311 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:26.663227 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:54:26.663311 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:26.663254 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:54:27.433237 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:27.433201 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" event={"ID":"dc3bdc4d-9167-4b54-94a5-a364b1deb668","Type":"ContainerStarted","Data":"51461cfeb010b7dce4ca44830bbb9578f0ac561529bcc65199f28e4a95285cc2"} Nov 04 04:54:27.594673 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:27.594620 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" podStartSLOduration=2.737877542 podStartE2EDuration="7.594606473s" totalImagesPullingTime="4.856728931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:54:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:54:27.592946385 +0000 UTC m=+509.304637526" watchObservedRunningTime="2026-11-04 04:54:27.594606473 +0000 UTC m=+509.306297624" Nov 04 04:54:28.339946 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:28.339911 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:28.341590 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:28.341556 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-psx2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Nov 04 04:54:28.341713 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:28.341613 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" podUID="dc3bdc4d-9167-4b54-94a5-a364b1deb668" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Nov 04 04:54:29.340421 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:29.340384 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-psx2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Nov 04 04:54:29.340942 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:29.340448 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" podUID="dc3bdc4d-9167-4b54-94a5-a364b1deb668" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Nov 04 04:54:30.340500 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:30.340464 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-psx2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Nov 04 04:54:30.340939 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:30.340517 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" podUID="dc3bdc4d-9167-4b54-94a5-a364b1deb668" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Nov 04 04:54:31.341113 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:31.341073 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-psx2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Nov 04 04:54:31.341503 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:31.341158 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" podUID="dc3bdc4d-9167-4b54-94a5-a364b1deb668" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Nov 04 04:54:32.340258 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:32.340218 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-psx2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Nov 04 04:54:32.340432 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:32.340278 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" podUID="dc3bdc4d-9167-4b54-94a5-a364b1deb668" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Nov 04 04:54:32.617061 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:32.616981 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:32.629455 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:32.629429 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:32.783690 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:32.783662 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:54:33.341117 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:33.341082 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-psx2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Nov 04 04:54:33.341292 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:33.341181 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" podUID="dc3bdc4d-9167-4b54-94a5-a364b1deb668" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Nov 04 04:54:33.468741 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:33.468711 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-8txv8" Nov 04 04:54:35.159223 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:35.159193 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:35.159714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:35.159529 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:35.160325 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:35.160296 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-psx2w" Nov 04 04:54:36.160901 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.160851 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb"] Nov 04 04:54:36.167396 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.167370 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.259655 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.259622 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb"] Nov 04 04:54:36.284239 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.284210 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.284395 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.284252 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.284395 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.284313 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.284395 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.284346 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/759be065-f27c-4886-8deb-1d56ced9139f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.284548 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.284398 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.284548 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.284417 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqcjn\" (UniqueName: \"kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-kube-api-access-rqcjn\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.284548 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.284448 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.284548 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.284531 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.284708 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.284580 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/759be065-f27c-4886-8deb-1d56ced9139f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.385796 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.385760 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.385969 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.385809 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.385969 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.385828 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/759be065-f27c-4886-8deb-1d56ced9139f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.385969 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.385908 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.385969 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.385946 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.386153 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.385976 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.386153 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.385998 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/759be065-f27c-4886-8deb-1d56ced9139f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.386153 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.386046 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.386153 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.386070 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqcjn\" (UniqueName: \"kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-kube-api-access-rqcjn\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.386503 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.386470 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.386622 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.386543 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.386622 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.386606 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/759be065-f27c-4886-8deb-1d56ced9139f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.386780 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.386756 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.386970 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.386953 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.389457 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.389435 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.389594 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.389578 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/759be065-f27c-4886-8deb-1d56ced9139f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.443201 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.443144 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.443307 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.443217 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqcjn\" (UniqueName: \"kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-kube-api-access-rqcjn\") pod \"maas-default-gateway-openshift-default-57f78bcd55-4t5gb\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.480659 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.480628 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:36.940939 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.940914 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb"] Nov 04 04:54:36.943469 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:54:36.943439 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod759be065_f27c_4886_8deb_1d56ced9139f.slice/crio-c332d48cf0f321cda1fb467a2eead53c62f833762aa0b1d122b27303915eee9e WatchSource:0}: Error finding container c332d48cf0f321cda1fb467a2eead53c62f833762aa0b1d122b27303915eee9e: Status 404 returned error can't find the container with id c332d48cf0f321cda1fb467a2eead53c62f833762aa0b1d122b27303915eee9e Nov 04 04:54:36.945607 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.945575 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:54:36.945664 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.945640 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:54:36.945697 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:36.945669 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:54:37.479584 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:37.479547 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" event={"ID":"759be065-f27c-4886-8deb-1d56ced9139f","Type":"ContainerStarted","Data":"057e5aeaa6a9004a02a8b161726dc2dfe6410090839437756b275499a85d53c4"} Nov 04 04:54:37.480115 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:37.479598 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" event={"ID":"759be065-f27c-4886-8deb-1d56ced9139f","Type":"ContainerStarted","Data":"c332d48cf0f321cda1fb467a2eead53c62f833762aa0b1d122b27303915eee9e"} Nov 04 04:54:37.480913 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:37.480891 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:37.482189 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:37.482168 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4t5gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Nov 04 04:54:37.482277 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:37.482210 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" podUID="759be065-f27c-4886-8deb-1d56ced9139f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Nov 04 04:54:37.594419 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:37.594365 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" podStartSLOduration=1.594350576 podStartE2EDuration="1.594350576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:54:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:54:37.592577531 +0000 UTC m=+519.304268671" watchObservedRunningTime="2026-11-04 04:54:37.594350576 +0000 UTC m=+519.306041715" Nov 04 04:54:38.480957 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:38.480914 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4t5gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Nov 04 04:54:38.481357 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:38.481011 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" podUID="759be065-f27c-4886-8deb-1d56ced9139f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Nov 04 04:54:39.481919 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:39.481861 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4t5gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Nov 04 04:54:39.482397 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:39.481944 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" podUID="759be065-f27c-4886-8deb-1d56ced9139f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Nov 04 04:54:40.481822 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:40.481781 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4t5gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Nov 04 04:54:40.482023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:40.481858 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" podUID="759be065-f27c-4886-8deb-1d56ced9139f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Nov 04 04:54:41.481061 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:41.481022 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4t5gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Nov 04 04:54:41.481264 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:41.481091 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" podUID="759be065-f27c-4886-8deb-1d56ced9139f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Nov 04 04:54:41.677330 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:41.677294 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:42.481209 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:42.481174 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4t5gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Nov 04 04:54:42.481391 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:42.481224 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" podUID="759be065-f27c-4886-8deb-1d56ced9139f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Nov 04 04:54:42.629140 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:42.629103 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:43.141928 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.141893 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:43.147902 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.147862 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-cpwbr"] Nov 04 04:54:43.151461 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.151443 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" Nov 04 04:54:43.166905 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.166852 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-metrics-certs\") pod \"ai-gateway-controller-55cd769764-cpwbr\" (UID: \"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8\") " pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" Nov 04 04:54:43.167074 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.167053 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtrx6\" (UniqueName: \"kubernetes.io/projected/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-kube-api-access-wtrx6\") pod \"ai-gateway-controller-55cd769764-cpwbr\" (UID: \"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8\") " pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" Nov 04 04:54:43.200282 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.200249 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-25qcs\"" Nov 04 04:54:43.246739 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.246698 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\"" Nov 04 04:54:43.246955 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.246766 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\"" Nov 04 04:54:43.248571 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.248546 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-cpwbr"] Nov 04 04:54:43.268132 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.268108 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-metrics-certs\") pod \"ai-gateway-controller-55cd769764-cpwbr\" (UID: \"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8\") " pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" Nov 04 04:54:43.268238 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.268212 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtrx6\" (UniqueName: \"kubernetes.io/projected/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-kube-api-access-wtrx6\") pod \"ai-gateway-controller-55cd769764-cpwbr\" (UID: \"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8\") " pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" Nov 04 04:54:43.270618 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.270593 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-metrics-certs\") pod \"ai-gateway-controller-55cd769764-cpwbr\" (UID: \"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8\") " pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" Nov 04 04:54:43.299345 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.299313 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:43.326264 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.326237 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtrx6\" (UniqueName: \"kubernetes.io/projected/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-kube-api-access-wtrx6\") pod \"ai-gateway-controller-55cd769764-cpwbr\" (UID: \"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8\") " pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" Nov 04 04:54:43.464232 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.464140 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" Nov 04 04:54:43.481245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.481212 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4t5gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Nov 04 04:54:43.481403 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.481291 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" podUID="759be065-f27c-4886-8deb-1d56ced9139f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Nov 04 04:54:43.831833 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:43.831642 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-cpwbr"] Nov 04 04:54:44.480971 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:44.480936 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-4t5gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Nov 04 04:54:44.481381 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:44.480996 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" podUID="759be065-f27c-4886-8deb-1d56ced9139f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Nov 04 04:54:44.513004 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:44.512969 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" event={"ID":"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8","Type":"ContainerStarted","Data":"b36de9d2b67f50e156a5f92f853393c53a1df25f8f61261defb466041e7fc202"} Nov 04 04:54:45.566635 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:45.566600 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:45.567111 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:45.566993 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:45.567984 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:45.567958 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 04:54:51.545351 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:51.545315 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" event={"ID":"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8","Type":"ContainerStarted","Data":"df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050"} Nov 04 04:54:51.545930 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:51.545909 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" Nov 04 04:54:51.655426 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:51.655371 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" podStartSLOduration=2.004859823 podStartE2EDuration="8.655358375s" totalImagesPullingTime="6.650498552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:54:51.654500004 +0000 UTC m=+533.366191143" watchObservedRunningTime="2026-11-04 04:54:51.655358375 +0000 UTC m=+533.367049516" Nov 04 04:54:53.346677 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.346642 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:53.516843 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.516810 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:54:53.755798 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.755766 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx"] Nov 04 04:54:53.759249 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.759223 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" Nov 04 04:54:53.813043 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.813014 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx"] Nov 04 04:54:53.857366 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.857338 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Nov 04 04:54:53.857366 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.857340 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jfbjs\"" Nov 04 04:54:53.869836 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.869809 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcnhs\" (UniqueName: \"kubernetes.io/projected/e1b03de5-92e0-4f77-9548-29ba4017ef0d-kube-api-access-lcnhs\") pod \"payload-pre-processing-7b98bb678c-7cvjx\" (UID: \"e1b03de5-92e0-4f77-9548-29ba4017ef0d\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" Nov 04 04:54:53.869993 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.869862 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b03de5-92e0-4f77-9548-29ba4017ef0d-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-7cvjx\" (UID: \"e1b03de5-92e0-4f77-9548-29ba4017ef0d\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" Nov 04 04:54:53.904644 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.904608 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-65r6s"] Nov 04 04:54:53.908117 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.908095 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-65r6s" Nov 04 04:54:53.962714 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.962676 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-65r6s"] Nov 04 04:54:53.970580 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.970546 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01b3ed18-6bc4-41c8-bad8-bc93b08b3746-plugins-config-volume\") pod \"payload-processing-7bb9489676-65r6s\" (UID: \"01b3ed18-6bc4-41c8-bad8-bc93b08b3746\") " pod="openshift-ingress/payload-processing-7bb9489676-65r6s" Nov 04 04:54:53.970757 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.970603 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcnhs\" (UniqueName: \"kubernetes.io/projected/e1b03de5-92e0-4f77-9548-29ba4017ef0d-kube-api-access-lcnhs\") pod \"payload-pre-processing-7b98bb678c-7cvjx\" (UID: \"e1b03de5-92e0-4f77-9548-29ba4017ef0d\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" Nov 04 04:54:53.970757 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.970668 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7444\" (UniqueName: \"kubernetes.io/projected/01b3ed18-6bc4-41c8-bad8-bc93b08b3746-kube-api-access-l7444\") pod \"payload-processing-7bb9489676-65r6s\" (UID: \"01b3ed18-6bc4-41c8-bad8-bc93b08b3746\") " pod="openshift-ingress/payload-processing-7bb9489676-65r6s" Nov 04 04:54:53.970757 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.970712 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b03de5-92e0-4f77-9548-29ba4017ef0d-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-7cvjx\" (UID: \"e1b03de5-92e0-4f77-9548-29ba4017ef0d\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" Nov 04 04:54:53.971401 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:53.971379 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b03de5-92e0-4f77-9548-29ba4017ef0d-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-7cvjx\" (UID: \"e1b03de5-92e0-4f77-9548-29ba4017ef0d\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" Nov 04 04:54:54.028640 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:54.028568 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcnhs\" (UniqueName: \"kubernetes.io/projected/e1b03de5-92e0-4f77-9548-29ba4017ef0d-kube-api-access-lcnhs\") pod \"payload-pre-processing-7b98bb678c-7cvjx\" (UID: \"e1b03de5-92e0-4f77-9548-29ba4017ef0d\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" Nov 04 04:54:54.071422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:54.071384 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" Nov 04 04:54:54.071657 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:54.071636 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01b3ed18-6bc4-41c8-bad8-bc93b08b3746-plugins-config-volume\") pod \"payload-processing-7bb9489676-65r6s\" (UID: \"01b3ed18-6bc4-41c8-bad8-bc93b08b3746\") " pod="openshift-ingress/payload-processing-7bb9489676-65r6s" Nov 04 04:54:54.071732 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:54.071694 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7444\" (UniqueName: \"kubernetes.io/projected/01b3ed18-6bc4-41c8-bad8-bc93b08b3746-kube-api-access-l7444\") pod \"payload-processing-7bb9489676-65r6s\" (UID: \"01b3ed18-6bc4-41c8-bad8-bc93b08b3746\") " pod="openshift-ingress/payload-processing-7bb9489676-65r6s" Nov 04 04:54:54.072328 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:54.072305 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01b3ed18-6bc4-41c8-bad8-bc93b08b3746-plugins-config-volume\") pod \"payload-processing-7bb9489676-65r6s\" (UID: \"01b3ed18-6bc4-41c8-bad8-bc93b08b3746\") " pod="openshift-ingress/payload-processing-7bb9489676-65r6s" Nov 04 04:54:54.134002 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:54.131215 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7444\" (UniqueName: \"kubernetes.io/projected/01b3ed18-6bc4-41c8-bad8-bc93b08b3746-kube-api-access-l7444\") pod \"payload-processing-7bb9489676-65r6s\" (UID: \"01b3ed18-6bc4-41c8-bad8-bc93b08b3746\") " pod="openshift-ingress/payload-processing-7bb9489676-65r6s" Nov 04 04:54:54.226606 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:54.226565 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-65r6s" Nov 04 04:54:54.444111 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:54.444079 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx"] Nov 04 04:54:54.446064 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:54:54.446034 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b03de5_92e0_4f77_9548_29ba4017ef0d.slice/crio-60928740c32a52e8a53f2bcc61ef95593c7ff5decce8cb599117c88ed15a0ff5 WatchSource:0}: Error finding container 60928740c32a52e8a53f2bcc61ef95593c7ff5decce8cb599117c88ed15a0ff5: Status 404 returned error can't find the container with id 60928740c32a52e8a53f2bcc61ef95593c7ff5decce8cb599117c88ed15a0ff5 Nov 04 04:54:54.559660 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:54.559572 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" event={"ID":"e1b03de5-92e0-4f77-9548-29ba4017ef0d","Type":"ContainerStarted","Data":"60928740c32a52e8a53f2bcc61ef95593c7ff5decce8cb599117c88ed15a0ff5"} Nov 04 04:54:54.598930 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:54.598902 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-65r6s"] Nov 04 04:54:54.600810 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:54:54.600785 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b3ed18_6bc4_41c8_bad8_bc93b08b3746.slice/crio-9d7044e9aa161279d05d8a0e666ed20dc03853f5f2cb1ec73285b22f843d90f1 WatchSource:0}: Error finding container 9d7044e9aa161279d05d8a0e666ed20dc03853f5f2cb1ec73285b22f843d90f1: Status 404 returned error can't find the container with id 9d7044e9aa161279d05d8a0e666ed20dc03853f5f2cb1ec73285b22f843d90f1 Nov 04 04:54:55.569314 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:55.569270 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-65r6s" event={"ID":"01b3ed18-6bc4-41c8-bad8-bc93b08b3746","Type":"ContainerStarted","Data":"9d7044e9aa161279d05d8a0e666ed20dc03853f5f2cb1ec73285b22f843d90f1"} Nov 04 04:54:55.896642 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:54:55.896526 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:54:58.589168 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:58.589131 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" event={"ID":"e1b03de5-92e0-4f77-9548-29ba4017ef0d","Type":"ContainerStarted","Data":"924a7aaa06ba59ba4140ad1dab7bdafca308977570d147241301ee9a9bd50144"} Nov 04 04:54:58.589667 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:58.589232 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" Nov 04 04:54:58.590638 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:58.590611 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-65r6s" event={"ID":"01b3ed18-6bc4-41c8-bad8-bc93b08b3746","Type":"ContainerStarted","Data":"e767a868f12d776154156f942280cf5f428ca0748f8cf1310370b6545b7d3ff0"} Nov 04 04:54:58.590793 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:58.590663 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bb9489676-65r6s" Nov 04 04:54:58.697409 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:58.697355 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" podStartSLOduration=2.030118391 podStartE2EDuration="5.69734173s" totalImagesPullingTime="3.667223339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:54:58.696813003 +0000 UTC m=+540.408504142" watchObservedRunningTime="2026-11-04 04:54:58.69734173 +0000 UTC m=+540.409032868" Nov 04 04:54:58.805848 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:54:58.805632 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bb9489676-65r6s" podStartSLOduration=2.284513444 podStartE2EDuration="5.805614196s" totalImagesPullingTime="3.521100752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:54:58.805219828 +0000 UTC m=+540.516911014" watchObservedRunningTime="2026-11-04 04:54:58.805614196 +0000 UTC m=+540.517305335" Nov 04 04:55:02.551770 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:02.551738 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" Nov 04 04:55:02.901458 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:02.901383 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7c679bcc4f-wjccp"] Nov 04 04:55:02.906062 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:02.906041 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" Nov 04 04:55:02.959050 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:02.959025 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7c679bcc4f-wjccp"] Nov 04 04:55:02.963285 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:02.963263 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfa90fbe-9a23-4420-be9b-5fee96dcdd44-metrics-certs\") pod \"ai-gateway-controller-7c679bcc4f-wjccp\" (UID: \"cfa90fbe-9a23-4420-be9b-5fee96dcdd44\") " pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" Nov 04 04:55:02.963364 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:02.963299 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp6kr\" (UniqueName: \"kubernetes.io/projected/cfa90fbe-9a23-4420-be9b-5fee96dcdd44-kube-api-access-cp6kr\") pod \"ai-gateway-controller-7c679bcc4f-wjccp\" (UID: \"cfa90fbe-9a23-4420-be9b-5fee96dcdd44\") " pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" Nov 04 04:55:03.064619 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:03.064584 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfa90fbe-9a23-4420-be9b-5fee96dcdd44-metrics-certs\") pod \"ai-gateway-controller-7c679bcc4f-wjccp\" (UID: \"cfa90fbe-9a23-4420-be9b-5fee96dcdd44\") " pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" Nov 04 04:55:03.064784 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:03.064626 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp6kr\" (UniqueName: \"kubernetes.io/projected/cfa90fbe-9a23-4420-be9b-5fee96dcdd44-kube-api-access-cp6kr\") pod \"ai-gateway-controller-7c679bcc4f-wjccp\" (UID: \"cfa90fbe-9a23-4420-be9b-5fee96dcdd44\") " pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" Nov 04 04:55:03.066961 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:03.066942 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfa90fbe-9a23-4420-be9b-5fee96dcdd44-metrics-certs\") pod \"ai-gateway-controller-7c679bcc4f-wjccp\" (UID: \"cfa90fbe-9a23-4420-be9b-5fee96dcdd44\") " pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" Nov 04 04:55:03.122555 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:03.122516 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp6kr\" (UniqueName: \"kubernetes.io/projected/cfa90fbe-9a23-4420-be9b-5fee96dcdd44-kube-api-access-cp6kr\") pod \"ai-gateway-controller-7c679bcc4f-wjccp\" (UID: \"cfa90fbe-9a23-4420-be9b-5fee96dcdd44\") " pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" Nov 04 04:55:03.218260 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:03.218185 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" Nov 04 04:55:03.580139 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:03.580112 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7c679bcc4f-wjccp"] Nov 04 04:55:03.583094 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:55:03.583064 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfa90fbe_9a23_4420_be9b_5fee96dcdd44.slice/crio-1f7bf7fa96253358305df76690e1b1a5090555056b631de6dbc503da1693275b WatchSource:0}: Error finding container 1f7bf7fa96253358305df76690e1b1a5090555056b631de6dbc503da1693275b: Status 404 returned error can't find the container with id 1f7bf7fa96253358305df76690e1b1a5090555056b631de6dbc503da1693275b Nov 04 04:55:03.613065 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:03.613036 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" event={"ID":"cfa90fbe-9a23-4420-be9b-5fee96dcdd44","Type":"ContainerStarted","Data":"1f7bf7fa96253358305df76690e1b1a5090555056b631de6dbc503da1693275b"} Nov 04 04:55:05.623481 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:05.623445 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" event={"ID":"cfa90fbe-9a23-4420-be9b-5fee96dcdd44","Type":"ContainerStarted","Data":"233803af0c724824c1192956dccc74d678aab7faf89e1e4ef8f668483fc7b9d0"} Nov 04 04:55:05.623989 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:05.623581 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" Nov 04 04:55:05.737746 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:05.737694 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" podStartSLOduration=2.663845036 podStartE2EDuration="3.73768125s" totalImagesPullingTime="1.073836214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:55:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:55:05.736017929 +0000 UTC m=+547.447709068" watchObservedRunningTime="2026-11-04 04:55:05.73768125 +0000 UTC m=+547.449372388" Nov 04 04:55:09.597734 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:09.597706 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bb9489676-65r6s" Nov 04 04:55:09.598186 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:09.597775 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7cvjx" Nov 04 04:55:16.631017 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:16.630912 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7c679bcc4f-wjccp" Nov 04 04:55:16.788900 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:16.788849 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-cpwbr"] Nov 04 04:55:16.789093 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:16.789069 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" podUID="4ed1e7aa-48cb-4667-955f-c7f9f3668ae8" containerName="manager" containerID="cri-o://df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050" gracePeriod=30 Nov 04 04:55:17.034038 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.034011 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" Nov 04 04:55:17.199189 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.199107 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-kube-api-access-wtrx6\" (UniqueName: \"kubernetes.io/projected/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-kube-api-access-wtrx6\") pod \"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8\" (UID: \"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8\") " Nov 04 04:55:17.199189 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.199157 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-metrics-certs\") pod \"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8\" (UID: \"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8\") " Nov 04 04:55:17.201281 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.201252 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-kube-api-access-wtrx6" pod "4ed1e7aa-48cb-4667-955f-c7f9f3668ae8" (UID: "4ed1e7aa-48cb-4667-955f-c7f9f3668ae8"). InnerVolumeSpecName "kube-api-access-wtrx6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 04:55:17.201375 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.201265 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-metrics-certs" pod "4ed1e7aa-48cb-4667-955f-c7f9f3668ae8" (UID: "4ed1e7aa-48cb-4667-955f-c7f9f3668ae8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 04:55:17.300448 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.300415 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtrx6\" (UniqueName: \"kubernetes.io/projected/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-kube-api-access-wtrx6\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:55:17.300448 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.300443 4374 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8-metrics-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 04:55:17.678073 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.678040 4374 generic.go:356] "Generic (PLEG): container finished" podID="4ed1e7aa-48cb-4667-955f-c7f9f3668ae8" containerID="df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050" exitCode=0 Nov 04 04:55:17.678473 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.678098 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" Nov 04 04:55:17.678473 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.678120 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" event={"ID":"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8","Type":"ContainerDied","Data":"df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050"} Nov 04 04:55:17.678473 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.678171 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-55cd769764-cpwbr" event={"ID":"4ed1e7aa-48cb-4667-955f-c7f9f3668ae8","Type":"ContainerDied","Data":"b36de9d2b67f50e156a5f92f853393c53a1df25f8f61261defb466041e7fc202"} Nov 04 04:55:17.678473 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.678193 4374 scope.go:122] "RemoveContainer" containerID="df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050" Nov 04 04:55:17.687622 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.687606 4374 scope.go:122] "RemoveContainer" containerID="df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050" Nov 04 04:55:17.687943 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:55:17.687921 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050\": container with ID starting with df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050 not found: ID does not exist" containerID="df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050" Nov 04 04:55:17.688018 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.687956 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050"} err="failed to get container status \"df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050\": rpc error: code = NotFound desc = could not find container \"df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050\": container with ID starting with df7a960f1a4706c3f3056466b8d501ed5179278b090843395e312d9beacd3050 not found: ID does not exist" Nov 04 04:55:17.831394 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.831361 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-cpwbr"] Nov 04 04:55:17.877843 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:17.877815 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-cpwbr"] Nov 04 04:55:18.381553 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:18.381520 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:18.410531 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:18.410493 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ed1e7aa-48cb-4667-955f-c7f9f3668ae8" path="/var/lib/kubelet/pods/4ed1e7aa-48cb-4667-955f-c7f9f3668ae8/volumes" Nov 04 04:55:19.304225 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:19.304180 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:20.384708 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:20.384682 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:20.528478 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:20.528435 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:21.541369 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.541334 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78"] Nov 04 04:55:21.541945 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.541927 4374 cpu_manager.go:414] "containerMap: removing container" podUID="4ed1e7aa-48cb-4667-955f-c7f9f3668ae8" containerName="manager" Nov 04 04:55:21.542030 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.541946 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ed1e7aa-48cb-4667-955f-c7f9f3668ae8" containerName="manager" Nov 04 04:55:21.542030 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.542021 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ed1e7aa-48cb-4667-955f-c7f9f3668ae8" containerName="manager" Nov 04 04:55:21.546771 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.546752 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.640505 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.640475 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.640654 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.640512 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmw7j\" (UniqueName: \"kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-kube-api-access-jmw7j\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.640654 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.640587 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.640739 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.640657 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.640775 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.640731 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.640775 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.640766 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.640852 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.640784 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.640927 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.640909 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.640963 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.640936 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.741894 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.741840 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78"] Nov 04 04:55:21.742171 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.742139 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.742171 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.742170 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmw7j\" (UniqueName: \"kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-kube-api-access-jmw7j\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.742294 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.742190 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.742294 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.742257 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.742420 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.742291 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.742420 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.742317 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.742420 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.742340 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.742567 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.742403 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.742567 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.742447 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.742805 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.742782 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.742935 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.742814 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-certs\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.743124 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.743099 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-credential-socket\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.745415 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.743118 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-data\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.745558 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.743825 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.745669 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.744988 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-envoy\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.745669 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.745303 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-podinfo\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.799682 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.799624 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmw7j\" (UniqueName: \"kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-kube-api-access-jmw7j\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.799777 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.799759 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-token\") pod \"maas-default-gateway-openshift-default-57f78bcd55-t9s78\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:21.860091 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:21.860064 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:22.021749 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:22.021694 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:22.170230 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:22.170132 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:22.276010 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:22.275979 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78"] Nov 04 04:55:22.308113 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:55:22.308088 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5340ea_6e5b_42ce_8800_23e4f25cc299.slice/crio-fa872d233d57ae2b3d3526ed2487684a9d6a572713989a014d1a68315ea2c33a WatchSource:0}: Error finding container fa872d233d57ae2b3d3526ed2487684a9d6a572713989a014d1a68315ea2c33a: Status 404 returned error can't find the container with id fa872d233d57ae2b3d3526ed2487684a9d6a572713989a014d1a68315ea2c33a Nov 04 04:55:22.310329 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:22.310299 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:55:22.310415 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:22.310365 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:55:22.310415 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:22.310397 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:55:22.702142 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:22.702100 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" event={"ID":"4f5340ea-6e5b-42ce-8800-23e4f25cc299","Type":"ContainerStarted","Data":"2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d"} Nov 04 04:55:22.702530 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:22.702172 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" event={"ID":"4f5340ea-6e5b-42ce-8800-23e4f25cc299","Type":"ContainerStarted","Data":"fa872d233d57ae2b3d3526ed2487684a9d6a572713989a014d1a68315ea2c33a"} Nov 04 04:55:22.835985 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:22.835930 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" podStartSLOduration=1.835915073 podStartE2EDuration="1.835915073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:55:22.834774143 +0000 UTC m=+564.546465283" watchObservedRunningTime="2026-11-04 04:55:22.835915073 +0000 UTC m=+564.547606212" Nov 04 04:55:22.860736 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:22.860704 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:23.862348 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:23.862317 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-t9s78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 04:55:23.862752 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:23.862365 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" podUID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 04:55:24.860656 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:24.860620 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-t9s78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 04:55:24.860957 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:24.860685 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" podUID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 04:55:25.853900 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:55:25.853849 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:55:25.861416 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:25.861390 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-t9s78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 04:55:25.861497 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:25.861444 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" podUID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 04:55:26.860571 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:26.860534 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-t9s78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 04:55:26.861008 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:26.860587 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" podUID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 04:55:27.861164 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:27.861124 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-t9s78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 04:55:27.861573 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:27.861192 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" podUID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 04:55:28.860549 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:28.860508 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-t9s78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 04:55:28.860750 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:28.860560 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" podUID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 04:55:29.861419 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:29.861381 4374 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-57f78bcd55-t9s78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 04:55:29.861807 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:29.861449 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" podUID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 04:55:30.871018 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:30.870986 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:30.871489 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:30.871412 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:30.872231 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:30.872209 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 04:55:42.024377 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:42.024304 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:52.771297 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:52.771265 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:52.914112 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:52.914080 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:53.374222 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.374190 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:53.581946 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.581909 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99"] Nov 04 04:55:53.585635 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.585612 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.629818 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.629731 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:53.645334 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.645304 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99"] Nov 04 04:55:53.680747 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.680719 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-c5bb6d-openshift-default-dockercfg-7x2zp\"" Nov 04 04:55:53.746470 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.746430 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.746668 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.746555 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-envoy\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.746668 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.746607 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-token\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.746668 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.746635 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istiod-ca-cert\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.746838 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.746718 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf5gl\" (UniqueName: \"kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-kube-api-access-jf5gl\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.746838 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.746751 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-certs\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.746838 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.746771 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-data\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.746838 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.746793 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-credential-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.746838 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.746813 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-podinfo\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.774141 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.774109 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:53.848257 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.848223 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf5gl\" (UniqueName: \"kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-kube-api-access-jf5gl\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.848257 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.848263 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-certs\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.848528 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.848293 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-data\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.848528 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.848324 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-credential-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.848528 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.848354 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-podinfo\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.848528 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.848414 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.848528 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.848473 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-envoy\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.848528 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.848505 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-token\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.848844 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.848532 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istiod-ca-cert\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.848844 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.848722 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-certs\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.848844 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.848749 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-data\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.849029 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.849008 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.849067 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.849016 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-credential-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.849408 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.849386 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istiod-ca-cert\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.850844 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.850812 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-envoy\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.850973 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.850961 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-podinfo\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.905694 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.905623 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf5gl\" (UniqueName: \"kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-kube-api-access-jf5gl\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.906274 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.906249 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-token\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:53.973077 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:53.973045 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:54.120720 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.120683 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:54.198679 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.198583 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:54.234241 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.234194 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp"] Nov 04 04:55:54.241166 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.241136 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.305286 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.305249 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-19bf5b-openshift-default-dockercfg-5x94f\"" Nov 04 04:55:54.306135 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.305980 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp"] Nov 04 04:55:54.334570 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.334543 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:54.354423 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.354390 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-socket\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.354554 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.354428 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-credential-socket\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.354554 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.354507 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8hsr\" (UniqueName: \"kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-kube-api-access-f8hsr\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.354645 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.354562 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istiod-ca-cert\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.354645 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.354580 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-certs\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.354755 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.354684 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-podinfo\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.354755 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.354726 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-data\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.354836 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.354769 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-envoy\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.354948 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.354930 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-token\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.455705 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.455631 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-socket\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.455705 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.455662 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-credential-socket\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.455705 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.455687 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8hsr\" (UniqueName: \"kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-kube-api-access-f8hsr\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.456002 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.455714 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istiod-ca-cert\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.456002 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.455741 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-certs\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.456002 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.455857 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-podinfo\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.456002 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.455939 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-data\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.456299 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.456022 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-envoy\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.456299 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.456179 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-token\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.456659 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.456634 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-credential-socket\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.456761 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.456669 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-certs\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.456761 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.456694 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-data\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.456895 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.456786 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-socket\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.457024 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.457006 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istiod-ca-cert\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.458359 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.458340 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-envoy\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.458502 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.458483 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-podinfo\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.485169 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.485146 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:54.517793 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.517758 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-token\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.517923 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.517909 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8hsr\" (UniqueName: \"kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-kube-api-access-f8hsr\") pod \"e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.527406 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.527386 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87"] Nov 04 04:55:54.532209 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.532187 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.558992 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.558969 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:54.581446 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.581418 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-204fa0-openshift-default-dockercfg-h2wrr\"" Nov 04 04:55:54.589979 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.589951 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87"] Nov 04 04:55:54.627840 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.627812 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:54.637300 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.637215 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99"] Nov 04 04:55:54.639894 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:55:54.639841 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12c89dac_0d97_4aea_b0fe_3e9a9502ae4a.slice/crio-c76c34c6439b6f3fbefd18460a18502b9b8e00b62a4ad93c39cb8dcf07959dbc WatchSource:0}: Error finding container c76c34c6439b6f3fbefd18460a18502b9b8e00b62a4ad93c39cb8dcf07959dbc: Status 404 returned error can't find the container with id c76c34c6439b6f3fbefd18460a18502b9b8e00b62a4ad93c39cb8dcf07959dbc Nov 04 04:55:54.642297 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.642259 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:55:54.642390 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.642345 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:55:54.642390 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.642384 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:55:54.658073 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.658049 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-podinfo\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.658177 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.658088 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-credential-socket\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.658275 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.658248 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-envoy\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.658374 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.658343 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52887\" (UniqueName: \"kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-kube-api-access-52887\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.658507 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.658384 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-data\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.658574 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.658502 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-socket\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.658574 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.658528 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9431eac2-29af-4785-8277-ab9bed9fd12d-istiod-ca-cert\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.658574 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.658564 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-certs\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.658721 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.658639 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-token\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.759361 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.759323 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52887\" (UniqueName: \"kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-kube-api-access-52887\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.759514 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.759384 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-data\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.759514 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.759498 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-socket\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.759654 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.759527 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9431eac2-29af-4785-8277-ab9bed9fd12d-istiod-ca-cert\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.759654 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.759568 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-certs\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.759654 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.759597 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-token\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.759654 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.759638 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-podinfo\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.759887 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.759674 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-credential-socket\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.759887 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.759734 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-envoy\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.759887 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.759783 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-data\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.760055 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.759884 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-socket\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.760055 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.759968 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-certs\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.760213 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.760190 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-credential-socket\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.760362 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.760340 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9431eac2-29af-4785-8277-ab9bed9fd12d-istiod-ca-cert\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.762402 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.762380 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-podinfo\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.762709 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.762695 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-envoy\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.820643 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.820616 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52887\" (UniqueName: \"kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-kube-api-access-52887\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.820991 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.820782 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-token\") pod \"e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.838193 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.838165 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:54.844278 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.844257 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:54.877123 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.877090 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" event={"ID":"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a","Type":"ContainerStarted","Data":"5b58fbb0a4c71a48e636d82ea221531ded1e8d8d43d97c6fbbeedda95bb3d109"} Nov 04 04:55:54.877325 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.877147 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" event={"ID":"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a","Type":"ContainerStarted","Data":"c76c34c6439b6f3fbefd18460a18502b9b8e00b62a4ad93c39cb8dcf07959dbc"} Nov 04 04:55:54.935045 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.933956 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp"] Nov 04 04:55:54.935045 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:55:54.934979 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b8bd7f2_adbf_4f67_9f2a_3a3998f0135b.slice/crio-3e02865e99a7992533803c314159714eb74360aab493de8cc94ea4dd715aaa47 WatchSource:0}: Error finding container 3e02865e99a7992533803c314159714eb74360aab493de8cc94ea4dd715aaa47: Status 404 returned error can't find the container with id 3e02865e99a7992533803c314159714eb74360aab493de8cc94ea4dd715aaa47 Nov 04 04:55:54.937961 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.937408 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:55:54.937961 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.937470 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:55:54.937961 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.937506 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:55:54.984847 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.984801 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:54.995720 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:54.995677 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" podStartSLOduration=1.995662885 podStartE2EDuration="1.995662885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:55:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:55:54.99385303 +0000 UTC m=+596.705544170" watchObservedRunningTime="2026-11-04 04:55:54.995662885 +0000 UTC m=+596.707354025" Nov 04 04:55:55.133164 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.133127 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:55.199332 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.199296 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:55.204476 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.204448 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:55.234272 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.234247 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87"] Nov 04 04:55:55.237104 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:55:55.237074 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9431eac2_29af_4785_8277_ab9bed9fd12d.slice/crio-2b6c93f5186b8eb9789bd61fc2bcbc64443ce095525d7cdc920899a738fc6033 WatchSource:0}: Error finding container 2b6c93f5186b8eb9789bd61fc2bcbc64443ce095525d7cdc920899a738fc6033: Status 404 returned error can't find the container with id 2b6c93f5186b8eb9789bd61fc2bcbc64443ce095525d7cdc920899a738fc6033 Nov 04 04:55:55.239169 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.239140 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:55:55.239239 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.239201 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:55:55.239239 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.239228 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:55:55.282256 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.282139 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:55.429245 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.429210 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:55.833130 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:55:55.833099 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:55:55.882967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.882927 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" event={"ID":"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b","Type":"ContainerStarted","Data":"90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1"} Nov 04 04:55:55.883151 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.882988 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" event={"ID":"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b","Type":"ContainerStarted","Data":"3e02865e99a7992533803c314159714eb74360aab493de8cc94ea4dd715aaa47"} Nov 04 04:55:55.884558 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.884534 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" event={"ID":"9431eac2-29af-4785-8277-ab9bed9fd12d","Type":"ContainerStarted","Data":"314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717"} Nov 04 04:55:55.884692 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.884617 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" event={"ID":"9431eac2-29af-4785-8277-ab9bed9fd12d","Type":"ContainerStarted","Data":"2b6c93f5186b8eb9789bd61fc2bcbc64443ce095525d7cdc920899a738fc6033"} Nov 04 04:55:55.884834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.884815 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:55.885653 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.885634 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:55:55.999215 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:55.999160 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" podStartSLOduration=1.999142477 podStartE2EDuration="1.999142477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:55:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:55:55.997711244 +0000 UTC m=+597.709402427" watchObservedRunningTime="2026-11-04 04:55:55.999142477 +0000 UTC m=+597.710833628" Nov 04 04:55:56.117269 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:56.117063 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" podStartSLOduration=2.117043773 podStartE2EDuration="2.117043773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:55:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:55:56.115718316 +0000 UTC m=+597.827409461" watchObservedRunningTime="2026-11-04 04:55:56.117043773 +0000 UTC m=+597.828734913" Nov 04 04:55:56.559921 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:56.559863 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:56.564943 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:56.564918 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:56.844584 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:56.844504 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:56.849879 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:56.849833 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:56.890296 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:56.890224 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:56.890296 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:56.890271 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:56.890834 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:56.890818 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:55:56.890905 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:56.890880 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:55:57.506934 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:57.506902 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:57.658144 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:57.658098 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:58.422526 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:58.422500 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:58.647475 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:58.647442 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:58.822944 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:58.822893 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:59.317277 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:59.317242 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:59.517746 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:59.517714 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:55:59.670139 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:55:59.670035 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:01.244048 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:01.244018 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:01.465164 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:01.465119 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:01.657125 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:01.657050 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:01.942442 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:01.942368 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:02.103063 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:02.103018 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:02.249556 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:02.249520 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:03.785751 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:03.785721 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:03.963134 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:03.963102 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:04.039031 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.038955 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp"] Nov 04 04:56:04.043808 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.043787 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.093009 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.092979 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-c5bb6d-serving-cert\"" Nov 04 04:56:04.095453 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.095432 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.095524 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.095480 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.095651 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.095630 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4msv\" (UniqueName: \"kubernetes.io/projected/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-kube-api-access-l4msv\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.095740 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.095723 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-metrics-certs\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.109682 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.109650 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:04.141216 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.141186 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp"] Nov 04 04:56:04.196945 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.196918 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-metrics-certs\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.197103 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.196963 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.197103 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.197002 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.197103 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.197077 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4msv\" (UniqueName: \"kubernetes.io/projected/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-kube-api-access-l4msv\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.197225 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:04.197147 4374 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-serving-cert: secret "maas-api-e2e-worker-w2-c5bb6d-serving-cert" not found Nov 04 04:56:04.197272 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:04.197259 4374 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-maas-api-tls podName:c0cec3be-0cf8-4043-bb2e-3cf776d2ec07 nodeName:}" failed. No retries permitted until 2026-11-04 04:56:04.697236368 +0000 UTC m=+606.408927503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-maas-api-tls") pod "maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" (UID: "c0cec3be-0cf8-4043-bb2e-3cf776d2ec07") : secret "maas-api-e2e-worker-w2-c5bb6d-serving-cert" not found Nov 04 04:56:04.197740 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.197704 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.199287 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.199266 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-metrics-certs\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.254982 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.254951 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4msv\" (UniqueName: \"kubernetes.io/projected/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-kube-api-access-l4msv\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.258260 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.258233 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:04.409148 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.409062 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:04.555109 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.555078 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:04.705988 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.705915 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.708486 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.708455 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:04.956810 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:04.956721 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:05.322783 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:05.322645 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp"] Nov 04 04:56:05.325186 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:56:05.325158 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0cec3be_0cf8_4043_bb2e_3cf776d2ec07.slice/crio-08afbfb4e753d922c566b0638b9efca78edfe219cc5ee0f4ce1271513478c24f WatchSource:0}: Error finding container 08afbfb4e753d922c566b0638b9efca78edfe219cc5ee0f4ce1271513478c24f: Status 404 returned error can't find the container with id 08afbfb4e753d922c566b0638b9efca78edfe219cc5ee0f4ce1271513478c24f Nov 04 04:56:05.931073 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:05.931042 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" event={"ID":"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07","Type":"ContainerStarted","Data":"08afbfb4e753d922c566b0638b9efca78edfe219cc5ee0f4ce1271513478c24f"} Nov 04 04:56:06.267461 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:06.267430 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:06.443822 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:06.443796 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:06.586303 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:06.586231 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:06.936863 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:06.936778 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" event={"ID":"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07","Type":"ContainerStarted","Data":"c6972de3525e2743dd0de4cb892f5762641c17f3135a7d0e33c339b06e8a39bb"} Nov 04 04:56:06.936863 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:06.936842 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:07.048934 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:07.048883 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" podStartSLOduration=3.148343878 podStartE2EDuration="4.048850919s" totalImagesPullingTime="900.507041ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:56:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:56:07.04628166 +0000 UTC m=+608.757972799" watchObservedRunningTime="2026-11-04 04:56:07.048850919 +0000 UTC m=+608.760542059" Nov 04 04:56:07.358923 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:07.358891 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:07.549404 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:07.549373 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:09.525492 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.525456 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx"] Nov 04 04:56:09.529419 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.529400 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:09.578372 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.578337 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-204e38-serving-cert\"" Nov 04 04:56:09.627002 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.626970 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx"] Nov 04 04:56:09.679145 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.679117 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdvlx\" (UniqueName: \"kubernetes.io/projected/fff61915-de2e-41b2-91c0-5e608c0b1fcb-kube-api-access-mdvlx\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:09.679310 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.679173 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-maas-api-tls\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:09.679310 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.679297 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-metrics-certs\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:09.679380 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.679367 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff61915-de2e-41b2-91c0-5e608c0b1fcb-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:09.726455 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.726424 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:09.784713 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.784626 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-maas-api-tls\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:09.785040 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.785014 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-metrics-certs\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:09.785215 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.785195 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff61915-de2e-41b2-91c0-5e608c0b1fcb-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:09.785593 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.785568 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdvlx\" (UniqueName: \"kubernetes.io/projected/fff61915-de2e-41b2-91c0-5e608c0b1fcb-kube-api-access-mdvlx\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:09.791315 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:09.786443 4374 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-serving-cert: secret "maas-api-e2e-worker-w3-204e38-serving-cert" not found Nov 04 04:56:09.791315 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:09.789702 4374 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-maas-api-tls podName:fff61915-de2e-41b2-91c0-5e608c0b1fcb nodeName:}" failed. No retries permitted until 2026-11-04 04:56:10.289677255 +0000 UTC m=+612.001368372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-maas-api-tls") pod "maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" (UID: "fff61915-de2e-41b2-91c0-5e608c0b1fcb") : secret "maas-api-e2e-worker-w3-204e38-serving-cert" not found Nov 04 04:56:09.791315 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.791252 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff61915-de2e-41b2-91c0-5e608c0b1fcb-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:09.796540 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.796516 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-metrics-certs\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:09.843641 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.843604 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdvlx\" (UniqueName: \"kubernetes.io/projected/fff61915-de2e-41b2-91c0-5e608c0b1fcb-kube-api-access-mdvlx\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:09.933297 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:09.933262 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:10.077522 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:10.077444 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:10.255296 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:10.255261 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:10.293362 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:10.293330 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-maas-api-tls\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:10.295668 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:10.295646 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-maas-api-tls\") pod \"maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:10.442256 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:10.442174 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:10.810626 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:10.810599 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx"] Nov 04 04:56:10.813509 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:56:10.813476 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff61915_de2e_41b2_91c0_5e608c0b1fcb.slice/crio-1a7cebe80668b103663feeff8bb66c92a42d3c9200ceb8cab8f7e295f2cfba7c WatchSource:0}: Error finding container 1a7cebe80668b103663feeff8bb66c92a42d3c9200ceb8cab8f7e295f2cfba7c: Status 404 returned error can't find the container with id 1a7cebe80668b103663feeff8bb66c92a42d3c9200ceb8cab8f7e295f2cfba7c Nov 04 04:56:10.957188 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:10.957151 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" event={"ID":"fff61915-de2e-41b2-91c0-5e608c0b1fcb","Type":"ContainerStarted","Data":"1a7cebe80668b103663feeff8bb66c92a42d3c9200ceb8cab8f7e295f2cfba7c"} Nov 04 04:56:11.963672 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:11.963638 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" event={"ID":"fff61915-de2e-41b2-91c0-5e608c0b1fcb","Type":"ContainerStarted","Data":"41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645"} Nov 04 04:56:11.964045 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:11.963716 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:12.124007 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:12.123905 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" podStartSLOduration=2.093481597 podStartE2EDuration="3.12388721s" totalImagesPullingTime="1.030405613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:56:12.122469426 +0000 UTC m=+613.834160565" watchObservedRunningTime="2026-11-04 04:56:12.12388721 +0000 UTC m=+613.835578350" Nov 04 04:56:12.393625 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:12.393548 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:56:12.947257 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:12.947230 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 04:56:14.991710 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:14.991651 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:15.379562 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:15.379476 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:15.575765 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:15.575715 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:17.973295 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:17.973268 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 04:56:18.182142 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:18.182080 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:18.384936 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:18.384850 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:19.409501 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.409464 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:56:19.759287 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.759249 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l"] Nov 04 04:56:19.763312 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.763291 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:19.812354 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.812324 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-a48f7e-serving-cert\"" Nov 04 04:56:19.861127 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.861096 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l"] Nov 04 04:56:19.887307 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.887276 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-metrics-certs\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:19.887506 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.887330 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:19.887506 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.887406 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-maas-api-tls\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:19.887628 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.887563 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svhvc\" (UniqueName: \"kubernetes.io/projected/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-kube-api-access-svhvc\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:19.964314 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.964282 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:19.988604 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.988564 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-metrics-certs\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:19.988797 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.988625 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:19.988797 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.988650 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-maas-api-tls\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:19.988797 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.988691 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svhvc\" (UniqueName: \"kubernetes.io/projected/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-kube-api-access-svhvc\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:19.988988 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:19.988791 4374 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-serving-cert: secret "maas-api-e2e-worker-w4-a48f7e-serving-cert" not found Nov 04 04:56:19.988988 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:19.988905 4374 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-maas-api-tls podName:ad8b2336-652f-4e73-ab74-4a6c69aa5bc5 nodeName:}" failed. No retries permitted until 2026-11-04 04:56:20.488856937 +0000 UTC m=+622.200548053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-maas-api-tls") pod "maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" (UID: "ad8b2336-652f-4e73-ab74-4a6c69aa5bc5") : secret "maas-api-e2e-worker-w4-a48f7e-serving-cert" not found Nov 04 04:56:19.989441 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.989419 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:19.991226 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:19.991205 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-metrics-certs\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:20.049031 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:20.048960 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svhvc\" (UniqueName: \"kubernetes.io/projected/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-kube-api-access-svhvc\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:20.140094 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:20.140058 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:20.411897 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:20.411810 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:20.495490 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:20.495460 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-maas-api-tls\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:20.497934 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:20.497912 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-maas-api-tls\") pod \"maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:20.676646 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:20.676570 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:21.047915 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:56:21.047885 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad8b2336_652f_4e73_ab74_4a6c69aa5bc5.slice/crio-63c0c5ed7bbb06cda857a22be02bb9d8ed22a7592417e751b81e702190d4b151 WatchSource:0}: Error finding container 63c0c5ed7bbb06cda857a22be02bb9d8ed22a7592417e751b81e702190d4b151: Status 404 returned error can't find the container with id 63c0c5ed7bbb06cda857a22be02bb9d8ed22a7592417e751b81e702190d4b151 Nov 04 04:56:21.066442 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:21.066416 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l"] Nov 04 04:56:22.018676 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:22.018648 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" event={"ID":"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5","Type":"ContainerStarted","Data":"63c0c5ed7bbb06cda857a22be02bb9d8ed22a7592417e751b81e702190d4b151"} Nov 04 04:56:23.025809 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:23.025776 4374 generic.go:356] "Generic (PLEG): container finished" podID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerID="5b58fbb0a4c71a48e636d82ea221531ded1e8d8d43d97c6fbbeedda95bb3d109" exitCode=0 Nov 04 04:56:23.026281 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:23.025847 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" event={"ID":"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a","Type":"ContainerDied","Data":"5b58fbb0a4c71a48e636d82ea221531ded1e8d8d43d97c6fbbeedda95bb3d109"} Nov 04 04:56:23.026281 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:23.026270 4374 scope.go:122] "RemoveContainer" containerID="5b58fbb0a4c71a48e636d82ea221531ded1e8d8d43d97c6fbbeedda95bb3d109" Nov 04 04:56:23.027770 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:23.027726 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:56:23.027895 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:23.027797 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" event={"ID":"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5","Type":"ContainerStarted","Data":"f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f"} Nov 04 04:56:23.027895 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:23.027830 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:56:23.028011 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:23.027896 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:56:23.028071 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:23.028037 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:23.279413 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:23.279298 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" podStartSLOduration=3.370429115 podStartE2EDuration="4.279284037s" totalImagesPullingTime="908.854922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:56:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:56:23.278086259 +0000 UTC m=+624.989777398" watchObservedRunningTime="2026-11-04 04:56:23.279284037 +0000 UTC m=+624.990975175" Nov 04 04:56:23.551720 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:23.551644 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:23.745765 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:23.745737 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:24.037627 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:24.037587 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" event={"ID":"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a","Type":"ContainerStarted","Data":"478f71e3be30523070819cdb23b35ccddde69b56045d9370b4d6d9a2f1fb3d4c"} Nov 04 04:56:24.199043 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:24.199010 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:56:24.199932 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:24.199905 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Nov 04 04:56:24.200063 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:24.199953 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Nov 04 04:56:24.845264 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:24.845222 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq"] Nov 04 04:56:24.852616 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:24.851331 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:24.901183 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:24.901157 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-19bf5b-serving-cert\"" Nov 04 04:56:24.923891 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:24.904304 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq"] Nov 04 04:56:24.950485 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:24.950457 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-maas-api-tls\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:24.950653 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:24.950589 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/751dc7b8-535d-4954-a0db-96f9dad7322c-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:24.950711 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:24.950688 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-metrics-certs\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:24.950764 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:24.950720 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thtfv\" (UniqueName: \"kubernetes.io/projected/751dc7b8-535d-4954-a0db-96f9dad7322c-kube-api-access-thtfv\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:25.048974 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.048941 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:25.052198 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.052167 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-metrics-certs\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:25.052335 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.052212 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thtfv\" (UniqueName: \"kubernetes.io/projected/751dc7b8-535d-4954-a0db-96f9dad7322c-kube-api-access-thtfv\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:25.052335 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.052320 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-maas-api-tls\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:25.052423 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.052408 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/751dc7b8-535d-4954-a0db-96f9dad7322c-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:25.052478 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:25.052451 4374 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-serving-cert: secret "maas-api-e2e-worker-w5-19bf5b-serving-cert" not found Nov 04 04:56:25.052540 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:25.052527 4374 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-maas-api-tls podName:751dc7b8-535d-4954-a0db-96f9dad7322c nodeName:}" failed. No retries permitted until 2026-11-04 04:56:25.552507048 +0000 UTC m=+627.264198179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-maas-api-tls") pod "maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" (UID: "751dc7b8-535d-4954-a0db-96f9dad7322c") : secret "maas-api-e2e-worker-w5-19bf5b-serving-cert" not found Nov 04 04:56:25.053052 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.053032 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/751dc7b8-535d-4954-a0db-96f9dad7322c-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:25.054428 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.054410 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-metrics-certs\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:25.131012 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.130939 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thtfv\" (UniqueName: \"kubernetes.io/projected/751dc7b8-535d-4954-a0db-96f9dad7322c-kube-api-access-thtfv\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:25.199550 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.199520 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Nov 04 04:56:25.199702 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.199583 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Nov 04 04:56:25.226904 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.226734 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:25.368165 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.368125 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:25.560249 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.560210 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-maas-api-tls\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:25.562661 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.562634 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-maas-api-tls\") pod \"maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:25.769043 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.769004 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:25.867397 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:25.867338 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:56:25.885602 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:25.885566 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:56:26.158512 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:26.158489 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq"] Nov 04 04:56:26.161035 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:56:26.161009 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod751dc7b8_535d_4954_a0db_96f9dad7322c.slice/crio-532df2471b04e12f095d5bee98f4227bc45934488c8fa13c2814e9d43387f444 WatchSource:0}: Error finding container 532df2471b04e12f095d5bee98f4227bc45934488c8fa13c2814e9d43387f444: Status 404 returned error can't find the container with id 532df2471b04e12f095d5bee98f4227bc45934488c8fa13c2814e9d43387f444 Nov 04 04:56:26.199112 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:26.199081 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Nov 04 04:56:26.199210 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:26.199122 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Nov 04 04:56:26.328894 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:26.328213 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:26.529666 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:26.529628 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:27.054177 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:27.054139 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" event={"ID":"751dc7b8-535d-4954-a0db-96f9dad7322c","Type":"ContainerStarted","Data":"532df2471b04e12f095d5bee98f4227bc45934488c8fa13c2814e9d43387f444"} Nov 04 04:56:27.199681 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:27.199658 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Nov 04 04:56:27.200059 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:27.199703 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Nov 04 04:56:28.016896 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:28.015140 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:56:28.061702 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:28.061670 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" event={"ID":"751dc7b8-535d-4954-a0db-96f9dad7322c","Type":"ContainerStarted","Data":"3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5"} Nov 04 04:56:28.061982 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:28.061962 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:28.174881 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:28.174815 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" podStartSLOduration=3.198208299 podStartE2EDuration="4.174799747s" totalImagesPullingTime="976.591448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:56:28.173245559 +0000 UTC m=+629.884936699" watchObservedRunningTime="2026-11-04 04:56:28.174799747 +0000 UTC m=+629.886490886" Nov 04 04:56:28.199797 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:28.199771 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Nov 04 04:56:28.200203 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:28.199815 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Nov 04 04:56:29.043291 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:29.043259 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 04:56:29.241087 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:29.241055 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:56:29.243268 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:29.243231 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 04:56:30.307684 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:30.307638 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:34.072385 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:34.072357 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 04:56:35.129756 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.129724 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z"] Nov 04 04:56:35.133845 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.133828 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.182542 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.182517 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-e62689-serving-cert\"" Nov 04 04:56:35.230893 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.230852 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z"] Nov 04 04:56:35.293043 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.293016 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-metrics-certs\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.293192 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.293079 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-maas-api-tls\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.293192 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.293180 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0075931-f3b3-4932-a18c-f2a172d6677b-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.293305 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.293233 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89x6h\" (UniqueName: \"kubernetes.io/projected/a0075931-f3b3-4932-a18c-f2a172d6677b-kube-api-access-89x6h\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.325121 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.325092 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:35.394823 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.394724 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89x6h\" (UniqueName: \"kubernetes.io/projected/a0075931-f3b3-4932-a18c-f2a172d6677b-kube-api-access-89x6h\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.394823 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.394800 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-metrics-certs\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.395128 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.394934 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-maas-api-tls\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.395128 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.395053 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0075931-f3b3-4932-a18c-f2a172d6677b-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.395539 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:35.395519 4374 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-serving-cert: secret "maas-api-e2e-worker-w1-e62689-serving-cert" not found Nov 04 04:56:35.395735 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:35.395712 4374 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-maas-api-tls podName:a0075931-f3b3-4932-a18c-f2a172d6677b nodeName:}" failed. No retries permitted until 2026-11-04 04:56:35.89569004 +0000 UTC m=+637.607381174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-maas-api-tls") pod "maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" (UID: "a0075931-f3b3-4932-a18c-f2a172d6677b") : secret "maas-api-e2e-worker-w1-e62689-serving-cert" not found Nov 04 04:56:35.397390 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.396183 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0075931-f3b3-4932-a18c-f2a172d6677b-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.401905 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.401882 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-metrics-certs\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.452257 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.452228 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89x6h\" (UniqueName: \"kubernetes.io/projected/a0075931-f3b3-4932-a18c-f2a172d6677b-kube-api-access-89x6h\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.497763 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.497735 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:35.639154 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.639120 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:35.902296 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.902262 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-maas-api-tls\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:35.904689 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:35.904669 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-maas-api-tls\") pod \"maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:36.046435 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:36.046403 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:36.413939 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:36.413843 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z"] Nov 04 04:56:36.416299 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:56:36.416259 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0075931_f3b3_4932_a18c_f2a172d6677b.slice/crio-42f1b7eb8b9b8fbfdc55027e0b52fe506b128811513ba30d68cd14696da16f6e WatchSource:0}: Error finding container 42f1b7eb8b9b8fbfdc55027e0b52fe506b128811513ba30d68cd14696da16f6e: Status 404 returned error can't find the container with id 42f1b7eb8b9b8fbfdc55027e0b52fe506b128811513ba30d68cd14696da16f6e Nov 04 04:56:36.459882 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:36.459845 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:56:37.109844 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:37.109812 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" event={"ID":"a0075931-f3b3-4932-a18c-f2a172d6677b","Type":"ContainerStarted","Data":"42f1b7eb8b9b8fbfdc55027e0b52fe506b128811513ba30d68cd14696da16f6e"} Nov 04 04:56:38.135138 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:38.135098 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" event={"ID":"a0075931-f3b3-4932-a18c-f2a172d6677b","Type":"ContainerStarted","Data":"250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df"} Nov 04 04:56:38.135614 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:38.135235 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:38.182450 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:38.182409 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:38.250256 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:38.250193 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" podStartSLOduration=2.35144821 podStartE2EDuration="3.250176115s" totalImagesPullingTime="898.727905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:56:38.249097814 +0000 UTC m=+639.960788953" watchObservedRunningTime="2026-11-04 04:56:38.250176115 +0000 UTC m=+639.961867280" Nov 04 04:56:38.763023 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:38.762968 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:38.935560 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:38.935512 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:39.079275 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:39.079182 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:40.834678 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:40.834351 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:40.993223 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:40.993175 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:41.139445 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:41.139331 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:42.097568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:42.097539 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:42.378444 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:42.378319 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:44.150732 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:44.150698 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 04:56:46.208475 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.208440 4374 generic.go:356] "Generic (PLEG): container finished" podID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerID="90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1" exitCode=0 Nov 04 04:56:46.208882 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.208501 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" event={"ID":"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b","Type":"ContainerDied","Data":"90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1"} Nov 04 04:56:46.208957 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.208860 4374 scope.go:122] "RemoveContainer" containerID="90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1" Nov 04 04:56:46.210134 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.210107 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:56:46.210188 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.210172 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:56:46.210232 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.210205 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:56:46.338449 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.338368 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq"] Nov 04 04:56:46.341763 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.341737 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:46.392069 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.392031 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-db8fae-serving-cert\"" Nov 04 04:56:46.421781 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.421704 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq"] Nov 04 04:56:46.429831 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.429804 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-maas-api-tls\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:46.430109 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.430063 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:46.430287 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.430112 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-metrics-certs\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:46.430287 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.430254 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6djx8\" (UniqueName: \"kubernetes.io/projected/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-kube-api-access-6djx8\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:46.531920 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.531882 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-maas-api-tls\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:46.532105 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.531982 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:46.532105 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.532021 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-metrics-certs\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:46.532105 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:46.532062 4374 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-serving-cert: secret "maas-api-e2e-worker-w6-db8fae-serving-cert" not found Nov 04 04:56:46.532240 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:46.532128 4374 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-maas-api-tls podName:8fca8cae-8eec-47a3-8efe-7a75bf7113c5 nodeName:}" failed. No retries permitted until 2026-11-04 04:56:47.032107209 +0000 UTC m=+648.743798347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-maas-api-tls") pod "maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" (UID: "8fca8cae-8eec-47a3-8efe-7a75bf7113c5") : secret "maas-api-e2e-worker-w6-db8fae-serving-cert" not found Nov 04 04:56:46.532240 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.532162 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6djx8\" (UniqueName: \"kubernetes.io/projected/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-kube-api-access-6djx8\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:46.532819 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.532790 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:46.534472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.534449 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-metrics-certs\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:46.535987 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.535964 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:46.589734 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.589701 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6djx8\" (UniqueName: \"kubernetes.io/projected/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-kube-api-access-6djx8\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:46.696096 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.696010 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:46.840749 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:46.840634 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:47.044598 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:47.044559 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-maas-api-tls\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:47.047001 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:47.046971 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-maas-api-tls\") pod \"maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:47.233494 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:47.233460 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" event={"ID":"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b","Type":"ContainerStarted","Data":"7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada"} Nov 04 04:56:47.254097 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:47.254068 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:47.560108 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:47.560017 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:56:47.560266 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:47.560179 4374 patch_prober.go:29] interesting pod/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 04:56:47.560266 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:47.560222 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 04:56:47.641819 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:47.641792 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq"] Nov 04 04:56:47.644091 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:56:47.644064 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fca8cae_8eec_47a3_8efe_7a75bf7113c5.slice/crio-05c457f44772f1d1e5f6a5966c548cbccf5c2bfe00fdd7a857df9c6c6c9a5c59 WatchSource:0}: Error finding container 05c457f44772f1d1e5f6a5966c548cbccf5c2bfe00fdd7a857df9c6c6c9a5c59: Status 404 returned error can't find the container with id 05c457f44772f1d1e5f6a5966c548cbccf5c2bfe00fdd7a857df9c6c6c9a5c59 Nov 04 04:56:48.239422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:48.239391 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" event={"ID":"8fca8cae-8eec-47a3-8efe-7a75bf7113c5","Type":"ContainerStarted","Data":"05c457f44772f1d1e5f6a5966c548cbccf5c2bfe00fdd7a857df9c6c6c9a5c59"} Nov 04 04:56:48.560166 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:48.560087 4374 patch_prober.go:29] interesting pod/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 04:56:48.560166 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:48.560142 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 04:56:49.245694 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:49.245659 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" event={"ID":"8fca8cae-8eec-47a3-8efe-7a75bf7113c5","Type":"ContainerStarted","Data":"d491c3f604368d4fabe98219170de4dcf5adda666cbb1fbe5443cf0555e6cda4"} Nov 04 04:56:49.246121 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:49.245849 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:49.358165 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:49.358112 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" podStartSLOduration=2.361615064 podStartE2EDuration="3.358096353s" totalImagesPullingTime="996.481289ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:56:49.355891941 +0000 UTC m=+651.067583071" watchObservedRunningTime="2026-11-04 04:56:49.358096353 +0000 UTC m=+651.069787506" Nov 04 04:56:49.560308 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:49.560231 4374 patch_prober.go:29] interesting pod/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 04:56:49.560308 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:49.560287 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 04:56:50.559702 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:50.559663 4374 patch_prober.go:29] interesting pod/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 04:56:50.560129 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:50.559722 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 04:56:51.560020 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:51.559991 4374 patch_prober.go:29] interesting pod/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 04:56:51.560382 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:51.560044 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 04:56:52.001363 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:52.001332 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:52.563923 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:52.563894 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:56:52.564420 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:52.564303 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:56:52.564915 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:52.564895 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 04:56:54.392106 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:54.392078 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:54.533240 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:54.533161 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:55.256341 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:55.256312 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 04:56:55.831760 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:56:55.831722 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:56:57.007116 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:57.007079 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:57.761161 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:57.761128 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:56:59.041428 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:59.041393 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:59.645230 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:59.645178 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:59.793568 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:59.793536 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:56:59.934886 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:56:59.934785 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:02.660942 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:02.660910 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:07.187986 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:07.187944 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:08.044206 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:08.044124 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:08.370532 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:08.370443 4374 generic.go:356] "Generic (PLEG): container finished" podID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerID="314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717" exitCode=0 Nov 04 04:57:08.370532 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:08.370519 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" event={"ID":"9431eac2-29af-4785-8277-ab9bed9fd12d","Type":"ContainerDied","Data":"314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717"} Nov 04 04:57:08.371047 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:08.370974 4374 scope.go:122] "RemoveContainer" containerID="314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717" Nov 04 04:57:08.372484 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:08.372449 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:57:08.372613 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:08.372533 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:57:08.372613 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:08.372575 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:57:08.544474 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:08.544437 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:09.124892 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:09.122371 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:09.267966 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:09.267924 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:09.390242 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:09.389827 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" event={"ID":"9431eac2-29af-4785-8277-ab9bed9fd12d","Type":"ContainerStarted","Data":"fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597"} Nov 04 04:57:09.419372 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:09.419337 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:09.845345 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:09.845312 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:57:09.846679 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:09.846639 4374 patch_prober.go:29] interesting pod/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 04:57:09.846805 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:09.846716 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 04:57:10.845633 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:10.845597 4374 patch_prober.go:29] interesting pod/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 04:57:10.846076 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:10.845658 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 04:57:11.844819 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:11.844781 4374 patch_prober.go:29] interesting pod/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 04:57:11.845041 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:11.844903 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 04:57:11.890413 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:11.890374 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:57:12.850356 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:12.850319 4374 patch_prober.go:29] interesting pod/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 04:57:12.850686 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:12.850656 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 04:57:13.597756 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:13.597725 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:13.844534 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:13.844486 4374 patch_prober.go:29] interesting pod/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 04:57:13.844705 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:13.844566 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 04:57:14.704320 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:14.704286 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:57:14.849218 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:14.849188 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:57:14.850219 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:14.850199 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 04:57:14.976964 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:14.976930 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:15.605748 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:15.605714 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:15.748551 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:15.748522 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:18.925955 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:18.925897 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:23.986573 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:23.986542 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:25.832589 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:57:25.832552 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:57:26.253566 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:26.253535 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:26.882934 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:26.882903 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:27.035938 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:27.035908 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:27.176967 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:27.176864 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:29.182803 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:29.182764 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:30.645391 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:30.645357 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:30.788837 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:30.788797 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:31.268944 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:31.268910 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:31.418522 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:31.418487 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:31.560612 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:31.560532 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:34.299860 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:34.299826 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:35.100582 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:35.100552 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:35.660693 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:35.660658 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:36.132190 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:36.132151 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:36.391296 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:36.391223 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:57:36.743472 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:36.743447 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:36.903039 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:36.903005 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:37.047747 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:37.047633 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:39.591079 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:39.591040 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:42.208237 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:42.208206 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:42.743112 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:42.743071 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:44.691290 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:44.691257 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:49.591715 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:49.591668 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:51.468323 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:51.468290 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:57:54.491047 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:54.491016 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:54.990938 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:54.990909 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:57:55.833766 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:57:55.833733 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:57:59.637111 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:57:59.637078 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:02.691349 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:02.691316 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:04.601604 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:04.601564 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:08.675198 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:08.675165 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:09.511521 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:09.511490 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:13.743391 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:13.743356 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:58:14.404918 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:14.404886 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:15.303549 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:15.303516 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:19.119463 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.119429 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6"] Nov 04 04:58:19.123135 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.123116 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.220647 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.220609 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-db8fae\"/\"openshift-service-ca.crt\"" Nov 04 04:58:19.220647 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.220620 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-db8fae\"/\"kube-root-ca.crt\"" Nov 04 04:58:19.220902 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.220670 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-db8fae\"/\"default-dockercfg-zjlmn\"" Nov 04 04:58:19.220902 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.220697 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-db8fae\"/\"simulator-w6-db8fae-kserve-self-signed-certs\"" Nov 04 04:58:19.221009 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.220916 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:19.223242 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.223218 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6"] Nov 04 04:58:19.228326 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.228308 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-home\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.228388 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.228373 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwbgt\" (UniqueName: \"kubernetes.io/projected/541835ff-1367-4ed6-8f6e-ddb66b753ff0-kube-api-access-nwbgt\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.228473 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.228455 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tls-certs\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.228522 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.228509 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tmp-dir\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.228618 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.228600 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-dshm\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.228653 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.228639 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-model-cache\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.329857 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.329826 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tmp-dir\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.330047 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.329906 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-dshm\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.330047 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.329934 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-model-cache\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.330047 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.329981 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-home\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.330207 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.330075 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwbgt\" (UniqueName: \"kubernetes.io/projected/541835ff-1367-4ed6-8f6e-ddb66b753ff0-kube-api-access-nwbgt\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.330207 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.330097 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tls-certs\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.330336 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.330308 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tmp-dir\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.330422 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.330399 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-model-cache\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.330483 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.330445 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-home\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.332432 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.332411 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-dshm\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.332662 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.332644 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tls-certs\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.379141 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.379082 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:19.387613 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.387580 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwbgt\" (UniqueName: \"kubernetes.io/projected/541835ff-1367-4ed6-8f6e-ddb66b753ff0-kube-api-access-nwbgt\") pod \"simulator-w6-db8fae-kserve-9768bb699-wdpv6\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.434662 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.434632 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:19.813767 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:19.813743 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6"] Nov 04 04:58:19.816107 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:58:19.816076 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod541835ff_1367_4ed6_8f6e_ddb66b753ff0.slice/crio-6ea223fc5fc29d7ac635714adde3013f768def53d64baeee02cbb2f36b818985 WatchSource:0}: Error finding container 6ea223fc5fc29d7ac635714adde3013f768def53d64baeee02cbb2f36b818985: Status 404 returned error can't find the container with id 6ea223fc5fc29d7ac635714adde3013f768def53d64baeee02cbb2f36b818985 Nov 04 04:58:20.759140 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:20.759103 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" event={"ID":"541835ff-1367-4ed6-8f6e-ddb66b753ff0","Type":"ContainerStarted","Data":"6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564"} Nov 04 04:58:20.759531 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:20.759150 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" event={"ID":"541835ff-1367-4ed6-8f6e-ddb66b753ff0","Type":"ContainerStarted","Data":"6ea223fc5fc29d7ac635714adde3013f768def53d64baeee02cbb2f36b818985"} Nov 04 04:58:20.880785 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:20.880735 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" podStartSLOduration=1.880717743 podStartE2EDuration="1.880717743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:58:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:58:20.879517497 +0000 UTC m=+742.591208628" watchObservedRunningTime="2026-11-04 04:58:20.880717743 +0000 UTC m=+742.592408885" Nov 04 04:58:21.434937 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:21.434906 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:24.371538 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:24.371507 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:25.836689 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:58:25.836651 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:58:29.394788 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:29.394755 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:29.456054 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:29.453065 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:29.469532 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:29.469508 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:29.816590 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:29.816561 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 04:58:34.294057 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:34.294020 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:37.772774 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:37.772733 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:58:39.203695 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:39.203643 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:43.085820 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:43.085786 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:44.103009 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:44.102980 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:48.972860 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:48.972777 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:49.919889 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:49.919827 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:53.888295 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:53.888251 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:55.446330 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.446295 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r"] Nov 04 04:58:55.450552 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.450528 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.499699 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.499663 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-204e38-openshift-default-dockercfg-ftp67\"" Nov 04 04:58:55.536268 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.536233 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-credential-socket\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.536442 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.536277 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bvmx\" (UniqueName: \"kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-kube-api-access-9bvmx\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.536523 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.536438 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-certs\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.536523 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.536475 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istiod-ca-cert\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.536523 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.536508 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-token\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.536683 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.536561 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-data\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.536683 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.536588 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-podinfo\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.536683 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.536641 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-socket\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.536683 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.536660 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-envoy\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.548394 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.548370 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r"] Nov 04 04:58:55.637220 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.637190 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bvmx\" (UniqueName: \"kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-kube-api-access-9bvmx\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.637373 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.637251 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-certs\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.637433 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.637409 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istiod-ca-cert\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.637489 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.637473 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-token\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.637544 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.637508 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-data\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.637544 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.637529 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-podinfo\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.637649 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.637553 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-certs\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.637649 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.637604 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-socket\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.637649 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.637636 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-envoy\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.637805 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.637743 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-credential-socket\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.637916 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.637897 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-data\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.638014 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.637991 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-socket\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.638116 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.638100 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-credential-socket\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.638235 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.638217 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istiod-ca-cert\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.639918 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.639899 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-envoy\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.639986 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.639971 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-podinfo\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.694589 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.694564 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-token\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.694696 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.694642 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bvmx\" (UniqueName: \"kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-kube-api-access-9bvmx\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.768087 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.768062 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:55.831890 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:55.830220 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:58:55.848792 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:58:55.848765 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:58:56.188283 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:56.188259 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r"] Nov 04 04:58:56.190640 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:58:56.190612 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b02e827_6e2d_4eee_898a_15f8c815eb3d.slice/crio-144767d439d001cfbd20b18d825f97ae5c88bd5360324039290dfb3469f1598c WatchSource:0}: Error finding container 144767d439d001cfbd20b18d825f97ae5c88bd5360324039290dfb3469f1598c: Status 404 returned error can't find the container with id 144767d439d001cfbd20b18d825f97ae5c88bd5360324039290dfb3469f1598c Nov 04 04:58:56.193010 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:56.192975 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:58:56.193073 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:56.193042 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:58:56.193115 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:56.193072 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:58:56.971940 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:56.971900 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" event={"ID":"7b02e827-6e2d-4eee-898a-15f8c815eb3d","Type":"ContainerStarted","Data":"2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c"} Nov 04 04:58:56.972322 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:56.971956 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" event={"ID":"7b02e827-6e2d-4eee-898a-15f8c815eb3d","Type":"ContainerStarted","Data":"144767d439d001cfbd20b18d825f97ae5c88bd5360324039290dfb3469f1598c"} Nov 04 04:58:57.135648 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:57.135593 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" podStartSLOduration=2.135578649 podStartE2EDuration="2.135578649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:58:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:58:57.134006054 +0000 UTC m=+778.845697205" watchObservedRunningTime="2026-11-04 04:58:57.135578649 +0000 UTC m=+778.847269788" Nov 04 04:58:57.768582 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:57.768544 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:58:57.769756 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:57.769731 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Nov 04 04:58:57.769897 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:57.769777 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" podUID="7b02e827-6e2d-4eee-898a-15f8c815eb3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Nov 04 04:58:58.769090 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:58.769052 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Nov 04 04:58:58.769510 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:58.769121 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" podUID="7b02e827-6e2d-4eee-898a-15f8c815eb3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Nov 04 04:58:58.829628 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:58.829580 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:58:59.769224 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:59.769187 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Nov 04 04:58:59.769707 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:58:59.769262 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" podUID="7b02e827-6e2d-4eee-898a-15f8c815eb3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Nov 04 04:59:00.769136 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:00.769100 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Nov 04 04:59:00.769317 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:00.769154 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" podUID="7b02e827-6e2d-4eee-898a-15f8c815eb3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Nov 04 04:59:01.768524 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:01.768483 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Nov 04 04:59:01.768713 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:01.768547 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" podUID="7b02e827-6e2d-4eee-898a-15f8c815eb3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Nov 04 04:59:02.891112 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:02.891067 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:59:02.891637 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:02.891397 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:59:02.892492 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:02.892468 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 04:59:03.797921 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:03.797841 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:08.695192 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:08.695161 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:13.077455 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:13.077424 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:13.695051 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:13.695015 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:18.599669 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:18.599593 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:20.898229 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:20.898193 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt"] Nov 04 04:59:20.902086 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:20.902062 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:20.950822 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:20.950786 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-model-cache\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:20.950987 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:20.950951 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-tmp-dir\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:20.951062 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:20.951047 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2n2m\" (UniqueName: \"kubernetes.io/projected/e25b779b-ecae-46cf-830a-4f5ededa9008-kube-api-access-l2n2m\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:20.951124 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:20.951072 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-e62689\"/\"kube-root-ca.crt\"" Nov 04 04:59:20.951124 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:20.951090 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-dshm\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:20.951229 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:20.951209 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-home\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:20.951273 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:20.951240 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e25b779b-ecae-46cf-830a-4f5ededa9008-tls-certs\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:20.991862 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:20.991829 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:20.999876 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:20.999843 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt"] Nov 04 04:59:21.000573 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.000550 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-e62689\"/\"default-dockercfg-mnjpr\"" Nov 04 04:59:21.000573 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.000550 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-e62689\"/\"premium-w1-e62689-kserve-self-signed-certs\"" Nov 04 04:59:21.060897 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.051488 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-e62689\"/\"openshift-service-ca.crt\"" Nov 04 04:59:21.060897 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.052190 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-model-cache\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.060897 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.052274 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-tmp-dir\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.060897 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.052354 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2n2m\" (UniqueName: \"kubernetes.io/projected/e25b779b-ecae-46cf-830a-4f5ededa9008-kube-api-access-l2n2m\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.060897 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.052398 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-dshm\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.060897 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.052497 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-home\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.060897 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.052522 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e25b779b-ecae-46cf-830a-4f5ededa9008-tls-certs\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.060897 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.052793 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-model-cache\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.060897 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.053860 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-home\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.060897 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.054004 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-tmp-dir\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.067372 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.067343 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-dshm\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.072902 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.072845 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e25b779b-ecae-46cf-830a-4f5ededa9008-tls-certs\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.112463 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.112435 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2n2m\" (UniqueName: \"kubernetes.io/projected/e25b779b-ecae-46cf-830a-4f5ededa9008-kube-api-access-l2n2m\") pod \"premium-w1-e62689-kserve-65bb6f74c7-7pqdt\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.215246 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.215174 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:21.620641 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:21.620615 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt"] Nov 04 04:59:21.622702 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:59:21.622675 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode25b779b_ecae_46cf_830a_4f5ededa9008.slice/crio-dce37e951edaa727fed1791829c9aa2dfb6da6020b73e29cab09f2c7d706b4d6 WatchSource:0}: Error finding container dce37e951edaa727fed1791829c9aa2dfb6da6020b73e29cab09f2c7d706b4d6: Status 404 returned error can't find the container with id dce37e951edaa727fed1791829c9aa2dfb6da6020b73e29cab09f2c7d706b4d6 Nov 04 04:59:22.098913 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:22.098836 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" event={"ID":"e25b779b-ecae-46cf-830a-4f5ededa9008","Type":"ContainerStarted","Data":"e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac"} Nov 04 04:59:22.098913 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:22.098900 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" event={"ID":"e25b779b-ecae-46cf-830a-4f5ededa9008","Type":"ContainerStarted","Data":"dce37e951edaa727fed1791829c9aa2dfb6da6020b73e29cab09f2c7d706b4d6"} Nov 04 04:59:22.208888 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:22.208672 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" podStartSLOduration=2.208658568 podStartE2EDuration="2.208658568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:59:22.208426331 +0000 UTC m=+803.920117470" watchObservedRunningTime="2026-11-04 04:59:22.208658568 +0000 UTC m=+803.920349707" Nov 04 04:59:22.215629 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:22.215598 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:22.322186 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:22.322134 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:22.463932 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:22.463819 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:23.120349 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:23.120316 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:59:23.500222 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:23.500191 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:25.832436 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:59:25.832395 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:59:28.425601 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:28.425571 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:28.569038 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:28.568957 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:31.215438 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:31.215401 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:31.228317 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:31.228290 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:32.171002 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:32.170968 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 04:59:33.300175 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:33.300143 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:37.801042 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:37.801005 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:38.189297 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:38.189209 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:40.787107 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:40.787063 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:43.196525 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:43.196491 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:43.449225 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:43.449138 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:48.094859 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:48.094823 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:52.468216 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:52.468183 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 04:59:53.006685 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:53.006616 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:53.273813 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:53.273735 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:55.831777 ip-10-0-140-242 kubenswrapper[4374]: E1104 04:59:55.831740 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 04:59:57.269836 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.269803 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk"] Nov 04 04:59:57.275546 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.275518 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.327024 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.326997 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-e62689-openshift-default-dockercfg-qvbw7\"" Nov 04 04:59:57.344680 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.344649 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt7dr\" (UniqueName: \"kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-kube-api-access-zt7dr\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.344818 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.344715 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/418c3a14-fbcb-4566-9009-5d4799d59593-istiod-ca-cert\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.344818 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.344790 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/418c3a14-fbcb-4566-9009-5d4799d59593-istio-podinfo\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.344917 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.344897 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-socket\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.344959 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.344920 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-data\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.344959 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.344940 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-envoy\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.345050 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.345036 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-credential-socket\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.345086 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.345072 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-certs\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.345122 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.345091 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-istio-token\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.375968 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.375935 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk"] Nov 04 04:59:57.446130 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.446097 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt7dr\" (UniqueName: \"kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-kube-api-access-zt7dr\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.446317 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.446185 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/418c3a14-fbcb-4566-9009-5d4799d59593-istiod-ca-cert\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.446317 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.446251 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/418c3a14-fbcb-4566-9009-5d4799d59593-istio-podinfo\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.446317 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.446278 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-socket\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.446317 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.446306 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-data\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.446562 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.446334 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-envoy\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.446562 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.446374 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-credential-socket\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.446562 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.446402 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-certs\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.446562 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.446431 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-istio-token\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.446950 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.446900 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-socket\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.447099 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.446985 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-certs\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.447099 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.447031 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-credential-socket\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.447099 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.447076 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/418c3a14-fbcb-4566-9009-5d4799d59593-istiod-ca-cert\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.447282 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.447187 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-data\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.448794 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.448769 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-envoy\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.448918 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.448880 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/418c3a14-fbcb-4566-9009-5d4799d59593-istio-podinfo\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.503325 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.503299 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-istio-token\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.503460 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.503381 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt7dr\" (UniqueName: \"kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-kube-api-access-zt7dr\") pod \"e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.590697 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.590612 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:57.980554 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:57.980525 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 04:59:58.011233 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:58.011209 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk"] Nov 04 04:59:58.013502 ip-10-0-140-242 kubenswrapper[4374]: W1104 04:59:58.013477 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod418c3a14_fbcb_4566_9009_5d4799d59593.slice/crio-b7b0f3478f7e7e1ae24dfbf0f82787fae67aedf12c9676508de6957caa0f130d WatchSource:0}: Error finding container b7b0f3478f7e7e1ae24dfbf0f82787fae67aedf12c9676508de6957caa0f130d: Status 404 returned error can't find the container with id b7b0f3478f7e7e1ae24dfbf0f82787fae67aedf12c9676508de6957caa0f130d Nov 04 04:59:58.015999 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:58.015963 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:59:58.016055 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:58.016034 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:59:58.016091 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:58.016080 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 04:59:58.317198 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:58.317101 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" event={"ID":"418c3a14-fbcb-4566-9009-5d4799d59593","Type":"ContainerStarted","Data":"524dc4def0b0202937f2810d18ff711396491448d891302bb523dcfe31134d37"} Nov 04 04:59:58.317198 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:58.317167 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" event={"ID":"418c3a14-fbcb-4566-9009-5d4799d59593","Type":"ContainerStarted","Data":"b7b0f3478f7e7e1ae24dfbf0f82787fae67aedf12c9676508de6957caa0f130d"} Nov 04 04:59:58.433266 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:58.433204 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" podStartSLOduration=1.433184662 podStartE2EDuration="1.433184662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:59:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:59:58.430089815 +0000 UTC m=+840.141780975" watchObservedRunningTime="2026-11-04 04:59:58.433184662 +0000 UTC m=+840.144875804" Nov 04 04:59:58.591241 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:58.591164 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 04:59:59.593518 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:59.593483 4374 patch_prober.go:29] interesting pod/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 04:59:59.593948 ip-10-0-140-242 kubenswrapper[4374]: I1104 04:59:59.593541 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" podUID="418c3a14-fbcb-4566-9009-5d4799d59593" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 05:00:00.591184 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:00.591149 4374 patch_prober.go:29] interesting pod/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 05:00:00.591364 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:00.591213 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" podUID="418c3a14-fbcb-4566-9009-5d4799d59593" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 05:00:01.591070 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:01.591036 4374 patch_prober.go:29] interesting pod/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 05:00:01.591496 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:01.591104 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" podUID="418c3a14-fbcb-4566-9009-5d4799d59593" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 05:00:01.799740 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:01.799687 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:02.591917 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:02.591853 4374 patch_prober.go:29] interesting pod/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 05:00:02.592316 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:02.591949 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" podUID="418c3a14-fbcb-4566-9009-5d4799d59593" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 05:00:02.897521 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:02.897426 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:03.591621 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:03.591583 4374 patch_prober.go:29] interesting pod/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Nov 04 05:00:03.591793 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:03.591650 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" podUID="418c3a14-fbcb-4566-9009-5d4799d59593" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Nov 04 05:00:04.869329 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:04.869290 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 05:00:04.869926 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:04.869620 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 05:00:04.872506 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:04.872481 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 05:00:07.917846 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:07.917813 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:10.171663 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:10.171629 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:12.984530 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:12.984496 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:18.087804 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:18.087768 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:19.991405 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:19.991370 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:23.151983 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:23.151936 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:25.621671 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.621638 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2"] Nov 04 05:00:25.625743 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.625720 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.676491 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.676465 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-a48f7e-openshift-default-dockercfg-tlcwj\"" Nov 04 05:00:25.698220 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.698188 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-istio-token\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.698389 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.698229 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww6cf\" (UniqueName: \"kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-kube-api-access-ww6cf\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.698389 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.698303 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-envoy\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.698389 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.698339 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-data\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.698389 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.698370 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51da6320-a68e-43f4-8a66-feff5075e34b-istio-podinfo\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.698630 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.698432 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-credential-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.698630 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.698462 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-certs\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.698741 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.698672 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.698741 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.698711 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51da6320-a68e-43f4-8a66-feff5075e34b-istiod-ca-cert\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.725402 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.725376 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2"] Nov 04 05:00:25.799715 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.799673 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-istio-token\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.799979 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.799728 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww6cf\" (UniqueName: \"kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-kube-api-access-ww6cf\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.799979 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.799767 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-envoy\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.800116 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.800092 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-data\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.800163 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.800150 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51da6320-a68e-43f4-8a66-feff5075e34b-istio-podinfo\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.800199 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.800179 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-credential-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.800199 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.800195 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-certs\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.800298 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.800285 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.800346 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.800310 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51da6320-a68e-43f4-8a66-feff5075e34b-istiod-ca-cert\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.800530 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.800499 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-data\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.800677 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.800544 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-credential-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.800756 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.800691 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-certs\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.800756 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.800704 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.801149 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.801127 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51da6320-a68e-43f4-8a66-feff5075e34b-istiod-ca-cert\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.802915 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.802896 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-envoy\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.803076 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.803059 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51da6320-a68e-43f4-8a66-feff5075e34b-istio-podinfo\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.833080 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:00:25.833047 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 05:00:25.861612 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.861582 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-istio-token\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.862598 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.862575 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww6cf\" (UniqueName: \"kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-kube-api-access-ww6cf\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:25.939906 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:25.939792 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:26.362679 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:26.362649 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2"] Nov 04 05:00:26.364687 ip-10-0-140-242 kubenswrapper[4374]: W1104 05:00:26.364655 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51da6320_a68e_43f4_8a66_feff5075e34b.slice/crio-6c03c2455a0131e102c78e80311ac5267b4700a08893edd1616441849527807b WatchSource:0}: Error finding container 6c03c2455a0131e102c78e80311ac5267b4700a08893edd1616441849527807b: Status 404 returned error can't find the container with id 6c03c2455a0131e102c78e80311ac5267b4700a08893edd1616441849527807b Nov 04 05:00:26.367119 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:26.367087 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:00:26.367194 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:26.367151 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:00:26.367194 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:26.367179 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:00:26.488181 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:26.488130 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" event={"ID":"51da6320-a68e-43f4-8a66-feff5075e34b","Type":"ContainerStarted","Data":"5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746"} Nov 04 05:00:26.488181 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:26.488185 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" event={"ID":"51da6320-a68e-43f4-8a66-feff5075e34b","Type":"ContainerStarted","Data":"6c03c2455a0131e102c78e80311ac5267b4700a08893edd1616441849527807b"} Nov 04 05:00:26.604738 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:26.604684 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" podStartSLOduration=1.604669013 podStartE2EDuration="1.604669013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:00:26.602724812 +0000 UTC m=+868.314415977" watchObservedRunningTime="2026-11-04 05:00:26.604669013 +0000 UTC m=+868.316360241" Nov 04 05:00:26.940730 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:26.940688 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:27.941377 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:27.941337 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 05:00:27.941797 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:27.941409 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" podUID="51da6320-a68e-43f4-8a66-feff5075e34b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 05:00:28.003802 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:28.003768 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:28.940733 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:28.940692 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 05:00:28.940919 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:28.940760 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" podUID="51da6320-a68e-43f4-8a66-feff5075e34b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 05:00:29.940339 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:29.940306 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 05:00:29.940734 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:29.940358 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" podUID="51da6320-a68e-43f4-8a66-feff5075e34b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 05:00:30.941037 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:30.941006 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 05:00:30.941445 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:30.941057 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" podUID="51da6320-a68e-43f4-8a66-feff5075e34b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 05:00:31.940445 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:31.940404 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 05:00:31.940623 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:31.940469 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" podUID="51da6320-a68e-43f4-8a66-feff5075e34b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 05:00:32.906440 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:32.906399 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:33.114203 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:33.112630 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:33.114203 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:33.113215 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:33.114441 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:33.114348 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:00:36.883933 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:36.883899 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 05:00:37.880440 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:37.880398 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:39.221609 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:39.221575 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78"] Nov 04 05:00:39.222161 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:39.221787 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" podUID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" containerName="istio-proxy" containerID="cri-o://2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d" gracePeriod=30 Nov 04 05:00:42.893130 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:42.893096 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:43.787203 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:43.787173 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:44.480578 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.480555 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 05:00:44.627282 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627195 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istiod-ca-cert\") pod \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " Nov 04 05:00:44.627282 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627243 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-token\" (UniqueName: \"kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-token\") pod \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " Nov 04 05:00:44.627528 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627284 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-credential-socket\") pod \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " Nov 04 05:00:44.627528 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627339 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-certs\") pod \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " Nov 04 05:00:44.627528 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627443 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-kube-api-access-jmw7j\" (UniqueName: \"kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-kube-api-access-jmw7j\") pod \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " Nov 04 05:00:44.627528 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627466 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-data\") pod \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " Nov 04 05:00:44.627528 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627487 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-podinfo\") pod \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " Nov 04 05:00:44.627528 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627517 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-envoy\") pod \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " Nov 04 05:00:44.627827 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627581 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-socket\") pod \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\" (UID: \"4f5340ea-6e5b-42ce-8800-23e4f25cc299\") " Nov 04 05:00:44.627827 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627706 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-certs" pod "4f5340ea-6e5b-42ce-8800-23e4f25cc299" (UID: "4f5340ea-6e5b-42ce-8800-23e4f25cc299"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:00:44.627827 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627726 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-credential-socket" pod "4f5340ea-6e5b-42ce-8800-23e4f25cc299" (UID: "4f5340ea-6e5b-42ce-8800-23e4f25cc299"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:00:44.627827 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627625 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istiod-ca-cert" pod "4f5340ea-6e5b-42ce-8800-23e4f25cc299" (UID: "4f5340ea-6e5b-42ce-8800-23e4f25cc299"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:00:44.627827 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.627805 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-data" pod "4f5340ea-6e5b-42ce-8800-23e4f25cc299" (UID: "4f5340ea-6e5b-42ce-8800-23e4f25cc299"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:00:44.628139 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.628089 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-socket" pod "4f5340ea-6e5b-42ce-8800-23e4f25cc299" (UID: "4f5340ea-6e5b-42ce-8800-23e4f25cc299"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:00:44.628231 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.628213 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:00:44.628295 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.628237 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:00:44.628295 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.628246 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:00:44.628295 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.628257 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:00:44.628295 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.628270 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:00:44.629910 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.629882 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-envoy" pod "4f5340ea-6e5b-42ce-8800-23e4f25cc299" (UID: "4f5340ea-6e5b-42ce-8800-23e4f25cc299"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:00:44.630023 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.629954 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-podinfo" pod "4f5340ea-6e5b-42ce-8800-23e4f25cc299" (UID: "4f5340ea-6e5b-42ce-8800-23e4f25cc299"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:00:44.630080 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.630058 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-token" pod "4f5340ea-6e5b-42ce-8800-23e4f25cc299" (UID: "4f5340ea-6e5b-42ce-8800-23e4f25cc299"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:00:44.630164 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.630148 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-kube-api-access-jmw7j" pod "4f5340ea-6e5b-42ce-8800-23e4f25cc299" (UID: "4f5340ea-6e5b-42ce-8800-23e4f25cc299"). InnerVolumeSpecName "kube-api-access-jmw7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:00:44.631648 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.631623 4374 generic.go:356] "Generic (PLEG): container finished" podID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" containerID="2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d" exitCode=0 Nov 04 05:00:44.631728 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.631681 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" Nov 04 05:00:44.631767 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.631680 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" event={"ID":"4f5340ea-6e5b-42ce-8800-23e4f25cc299","Type":"ContainerDied","Data":"2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d"} Nov 04 05:00:44.631812 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.631800 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78" event={"ID":"4f5340ea-6e5b-42ce-8800-23e4f25cc299","Type":"ContainerDied","Data":"fa872d233d57ae2b3d3526ed2487684a9d6a572713989a014d1a68315ea2c33a"} Nov 04 05:00:44.631850 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.631827 4374 scope.go:122] "RemoveContainer" containerID="2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d" Nov 04 05:00:44.658673 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.658646 4374 scope.go:122] "RemoveContainer" containerID="2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d" Nov 04 05:00:44.659066 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:00:44.659038 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d\": container with ID starting with 2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d not found: ID does not exist" containerID="2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d" Nov 04 05:00:44.659170 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.659071 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d"} err="failed to get container status \"2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d\": rpc error: code = NotFound desc = could not find container \"2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d\": container with ID starting with 2699ab67067ad172fb2bd5d0c8be28f7b4754343cde963ad5a96b531d833a33d not found: ID does not exist" Nov 04 05:00:44.730391 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.730355 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:00:44.730391 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.730391 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmw7j\" (UniqueName: \"kubernetes.io/projected/4f5340ea-6e5b-42ce-8800-23e4f25cc299-kube-api-access-jmw7j\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:00:44.730583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.730409 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:00:44.730583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.730424 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f5340ea-6e5b-42ce-8800-23e4f25cc299-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:00:44.798362 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.798332 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78"] Nov 04 05:00:44.803425 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:44.803398 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-t9s78"] Nov 04 05:00:46.418223 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:46.418191 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" path="/var/lib/kubelet/pods/4f5340ea-6e5b-42ce-8800-23e4f25cc299/volumes" Nov 04 05:00:47.790043 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:47.790010 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:52.703620 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:52.703587 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:55.832600 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:00:55.832549 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 05:00:57.617193 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:57.617157 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:00:59.401763 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:00:59.401728 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:00.847021 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:00.846980 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:02.052321 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:02.052289 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:02.198249 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:02.198217 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:02.689930 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:02.689891 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:02.830541 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:02.830508 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:05.360745 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:05.360714 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:06.002995 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:06.002965 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:07.598278 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:07.598248 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:08.923379 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:08.923348 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:09.068090 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:09.068042 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:09.210779 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:09.210686 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:09.400781 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:09.400748 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 05:01:10.413527 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:10.413493 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:12.090577 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:12.090542 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:12.525579 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:12.525545 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:14.034321 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:14.034288 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt"] Nov 04 05:01:14.130279 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:14.130249 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:16.597294 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:16.597255 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:17.420631 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:17.420598 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:17.985086 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:17.985055 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:18.689153 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:18.689112 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:20.398950 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:20.398922 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:22.681324 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:22.681288 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:22.940254 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:22.940174 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:22.950634 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:22.950604 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6"] Nov 04 05:01:23.086311 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:23.086256 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:24.403106 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:24.403075 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:24.547540 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:24.547510 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:25.004778 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:25.004746 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:25.838853 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:01:25.838814 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 05:01:27.232180 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.232147 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:27.578037 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.577964 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:27.612644 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.612612 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr"] Nov 04 05:01:27.613177 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.613152 4374 cpu_manager.go:414] "containerMap: removing container" podUID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" containerName="istio-proxy" Nov 04 05:01:27.613177 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.613179 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" containerName="istio-proxy" Nov 04 05:01:27.613265 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.613248 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f5340ea-6e5b-42ce-8800-23e4f25cc299" containerName="istio-proxy" Nov 04 05:01:27.623046 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.623017 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.656597 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.656558 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-credential-socket\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.656998 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.656964 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-token\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.657146 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.657055 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-data\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.657146 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.657090 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-certs\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.657296 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.657193 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-socket\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.657296 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.657247 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-envoy\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.657411 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.657295 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-podinfo\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.657411 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.657322 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istiod-ca-cert\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.657411 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.657353 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmk9h\" (UniqueName: \"kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-kube-api-access-qmk9h\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.674402 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.674368 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr"] Nov 04 05:01:27.717705 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.717677 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-db8fae-openshift-default-dockercfg-l9grm\"" Nov 04 05:01:27.758416 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.758388 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-socket\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.758563 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.758432 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-envoy\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.758563 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.758470 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-podinfo\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.758563 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.758488 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istiod-ca-cert\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.758563 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.758505 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmk9h\" (UniqueName: \"kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-kube-api-access-qmk9h\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.758789 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.758649 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-credential-socket\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.758789 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.758746 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-token\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.758934 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.758790 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-socket\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.758934 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.758823 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-data\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.758934 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.758856 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-certs\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.759176 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.759151 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-data\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.759245 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.759219 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istiod-ca-cert\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.759300 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.759266 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-credential-socket\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.759300 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.759268 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-certs\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.760924 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.760901 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-envoy\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.761055 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.760997 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-podinfo\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.817375 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.817339 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-token\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.817665 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.817645 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmk9h\" (UniqueName: \"kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-kube-api-access-qmk9h\") pod \"e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:27.945640 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:27.945554 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:28.197574 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:28.195607 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:28.380369 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:28.380340 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr"] Nov 04 05:01:28.382400 ip-10-0-140-242 kubenswrapper[4374]: W1104 05:01:28.382373 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf269a2cb_cd4c_4f54_bbc4_d4a565f5d7da.slice/crio-6df3bebbbc375c89561d30c02dd92a8caea199a39ad5153f7228ac9cbc061c82 WatchSource:0}: Error finding container 6df3bebbbc375c89561d30c02dd92a8caea199a39ad5153f7228ac9cbc061c82: Status 404 returned error can't find the container with id 6df3bebbbc375c89561d30c02dd92a8caea199a39ad5153f7228ac9cbc061c82 Nov 04 05:01:28.384910 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:28.384851 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:01:28.385013 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:28.384960 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:01:28.385013 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:28.385004 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:01:28.907819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:28.907777 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" event={"ID":"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da","Type":"ContainerStarted","Data":"1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863"} Nov 04 05:01:28.907819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:28.907827 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" event={"ID":"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da","Type":"ContainerStarted","Data":"6df3bebbbc375c89561d30c02dd92a8caea199a39ad5153f7228ac9cbc061c82"} Nov 04 05:01:28.946325 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:28.946293 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:29.054958 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.054919 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" podUID="e25b779b-ecae-46cf-830a-4f5ededa9008" containerName="main" containerID="cri-o://e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac" gracePeriod=45 Nov 04 05:01:29.069929 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.069851 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" podStartSLOduration=2.069833037 podStartE2EDuration="2.069833037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:01:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:01:29.066829442 +0000 UTC m=+930.778520594" watchObservedRunningTime="2026-11-04 05:01:29.069833037 +0000 UTC m=+930.781524178" Nov 04 05:01:29.161137 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.161052 4374 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" podUID="e25b779b-ecae-46cf-830a-4f5ededa9008" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": dial tcp 10.133.0.49:8000: connect: connection refused" Nov 04 05:01:29.168624 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.168598 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:29.337911 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.337417 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:29.372615 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.372585 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 05:01:29.486586 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.486557 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e25b779b-ecae-46cf-830a-4f5ededa9008-kube-api-access-l2n2m\" (UniqueName: \"kubernetes.io/projected/e25b779b-ecae-46cf-830a-4f5ededa9008-kube-api-access-l2n2m\") pod \"e25b779b-ecae-46cf-830a-4f5ededa9008\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " Nov 04 05:01:29.487054 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.486609 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-home\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-home\") pod \"e25b779b-ecae-46cf-830a-4f5ededa9008\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " Nov 04 05:01:29.487054 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.486629 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-dshm\") pod \"e25b779b-ecae-46cf-830a-4f5ededa9008\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " Nov 04 05:01:29.487054 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.486674 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-model-cache\") pod \"e25b779b-ecae-46cf-830a-4f5ededa9008\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " Nov 04 05:01:29.487054 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.486723 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-tmp-dir\") pod \"e25b779b-ecae-46cf-830a-4f5ededa9008\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " Nov 04 05:01:29.487054 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.486764 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e25b779b-ecae-46cf-830a-4f5ededa9008-tls-certs\" (UniqueName: \"kubernetes.io/secret/e25b779b-ecae-46cf-830a-4f5ededa9008-tls-certs\") pod \"e25b779b-ecae-46cf-830a-4f5ededa9008\" (UID: \"e25b779b-ecae-46cf-830a-4f5ededa9008\") " Nov 04 05:01:29.487717 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.487689 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-model-cache" pod "e25b779b-ecae-46cf-830a-4f5ededa9008" (UID: "e25b779b-ecae-46cf-830a-4f5ededa9008"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:01:29.491016 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.487838 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-home" pod "e25b779b-ecae-46cf-830a-4f5ededa9008" (UID: "e25b779b-ecae-46cf-830a-4f5ededa9008"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:01:29.491016 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.490086 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e25b779b-ecae-46cf-830a-4f5ededa9008-kube-api-access-l2n2m" pod "e25b779b-ecae-46cf-830a-4f5ededa9008" (UID: "e25b779b-ecae-46cf-830a-4f5ededa9008"). InnerVolumeSpecName "kube-api-access-l2n2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:01:29.491016 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.490539 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-tmp-dir" pod "e25b779b-ecae-46cf-830a-4f5ededa9008" (UID: "e25b779b-ecae-46cf-830a-4f5ededa9008"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:01:29.496815 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.492500 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e25b779b-ecae-46cf-830a-4f5ededa9008-tls-certs" pod "e25b779b-ecae-46cf-830a-4f5ededa9008" (UID: "e25b779b-ecae-46cf-830a-4f5ededa9008"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:01:29.496815 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.495223 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-dshm" pod "e25b779b-ecae-46cf-830a-4f5ededa9008" (UID: "e25b779b-ecae-46cf-830a-4f5ededa9008"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:01:29.588470 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.588434 4374 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-model-cache\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:29.588470 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.588465 4374 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-tmp-dir\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:29.588470 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.588474 4374 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e25b779b-ecae-46cf-830a-4f5ededa9008-tls-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:29.588689 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.588484 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2n2m\" (UniqueName: \"kubernetes.io/projected/e25b779b-ecae-46cf-830a-4f5ededa9008-kube-api-access-l2n2m\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:29.588689 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.588493 4374 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-home\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:29.588689 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.588500 4374 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e25b779b-ecae-46cf-830a-4f5ededa9008-dshm\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:29.755645 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.755560 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:29.903605 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.903577 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:29.935683 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.935648 4374 generic.go:356] "Generic (PLEG): container finished" podID="e25b779b-ecae-46cf-830a-4f5ededa9008" containerID="e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac" exitCode=0 Nov 04 05:01:29.935822 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.935696 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" Nov 04 05:01:29.935822 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.935724 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" event={"ID":"e25b779b-ecae-46cf-830a-4f5ededa9008","Type":"ContainerDied","Data":"e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac"} Nov 04 05:01:29.935822 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.935764 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt" event={"ID":"e25b779b-ecae-46cf-830a-4f5ededa9008","Type":"ContainerDied","Data":"dce37e951edaa727fed1791829c9aa2dfb6da6020b73e29cab09f2c7d706b4d6"} Nov 04 05:01:29.935822 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.935785 4374 scope.go:122] "RemoveContainer" containerID="e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac" Nov 04 05:01:29.951271 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.948696 4374 scope.go:122] "RemoveContainer" containerID="e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac" Nov 04 05:01:29.951271 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:01:29.949036 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac\": container with ID starting with e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac not found: ID does not exist" containerID="e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac" Nov 04 05:01:29.951271 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.949098 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac"} err="failed to get container status \"e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac\": rpc error: code = NotFound desc = could not find container \"e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac\": container with ID starting with e5157517b128812b020f7a4021c8da3e863ed67d909e86dab14c7d046f7621ac not found: ID does not exist" Nov 04 05:01:29.961301 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.957461 4374 patch_prober.go:29] interesting pod/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Nov 04 05:01:29.961301 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:29.957529 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" podUID="f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Nov 04 05:01:30.089273 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:30.089193 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt"] Nov 04 05:01:30.095464 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:30.095432 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-e62689/premium-w1-e62689-kserve-65bb6f74c7-7pqdt"] Nov 04 05:01:30.440151 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:30.440065 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e25b779b-ecae-46cf-830a-4f5ededa9008" path="/var/lib/kubelet/pods/e25b779b-ecae-46cf-830a-4f5ededa9008/volumes" Nov 04 05:01:30.872090 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:30.872010 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:30.947894 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:30.947350 4374 patch_prober.go:29] interesting pod/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Nov 04 05:01:30.947894 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:30.947433 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" podUID="f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Nov 04 05:01:31.946525 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:31.946490 4374 patch_prober.go:29] interesting pod/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Nov 04 05:01:31.946927 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:31.946551 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" podUID="f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Nov 04 05:01:32.432754 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:32.432718 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:32.575036 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:32.575008 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:32.946690 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:32.946657 4374 patch_prober.go:29] interesting pod/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Nov 04 05:01:32.947122 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:32.946714 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" podUID="f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Nov 04 05:01:33.204016 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:33.203928 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:33.450013 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:33.449971 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:33.947404 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:33.947340 4374 patch_prober.go:29] interesting pod/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Nov 04 05:01:33.948121 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:33.948008 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" podUID="f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Nov 04 05:01:34.946591 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:34.946557 4374 patch_prober.go:29] interesting pod/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Nov 04 05:01:34.946775 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:34.946621 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" podUID="f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Nov 04 05:01:35.103038 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:35.103007 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:35.246567 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:35.246534 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:35.925494 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:35.925463 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:35.951022 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:35.950994 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:35.951391 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:35.951364 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:35.952025 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:35.952009 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:01:36.065790 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:36.065755 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:37.971611 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:37.971568 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" podUID="541835ff-1367-4ed6-8f6e-ddb66b753ff0" containerName="main" containerID="cri-o://6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564" gracePeriod=45 Nov 04 05:01:38.239510 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.239487 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 05:01:38.320271 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.320239 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:38.359516 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.359485 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tmp-dir\") pod \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " Nov 04 05:01:38.359702 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.359550 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tls-certs\" (UniqueName: \"kubernetes.io/secret/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tls-certs\") pod \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " Nov 04 05:01:38.359702 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.359617 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/541835ff-1367-4ed6-8f6e-ddb66b753ff0-kube-api-access-nwbgt\" (UniqueName: \"kubernetes.io/projected/541835ff-1367-4ed6-8f6e-ddb66b753ff0-kube-api-access-nwbgt\") pod \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " Nov 04 05:01:38.359834 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.359784 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tmp-dir" pod "541835ff-1367-4ed6-8f6e-ddb66b753ff0" (UID: "541835ff-1367-4ed6-8f6e-ddb66b753ff0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:01:38.360141 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.360118 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-dshm\") pod \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " Nov 04 05:01:38.360266 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.360182 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-model-cache\") pod \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " Nov 04 05:01:38.360266 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.360224 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-home\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-home\") pod \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\" (UID: \"541835ff-1367-4ed6-8f6e-ddb66b753ff0\") " Nov 04 05:01:38.360908 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.360693 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-model-cache" pod "541835ff-1367-4ed6-8f6e-ddb66b753ff0" (UID: "541835ff-1367-4ed6-8f6e-ddb66b753ff0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:01:38.360908 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.360806 4374 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-model-cache\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:38.360908 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.360826 4374 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tmp-dir\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:38.361163 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.361131 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-home" pod "541835ff-1367-4ed6-8f6e-ddb66b753ff0" (UID: "541835ff-1367-4ed6-8f6e-ddb66b753ff0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:01:38.362546 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.362510 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-dshm" pod "541835ff-1367-4ed6-8f6e-ddb66b753ff0" (UID: "541835ff-1367-4ed6-8f6e-ddb66b753ff0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:01:38.362646 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.362594 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tls-certs" pod "541835ff-1367-4ed6-8f6e-ddb66b753ff0" (UID: "541835ff-1367-4ed6-8f6e-ddb66b753ff0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:01:38.363787 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.363755 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/541835ff-1367-4ed6-8f6e-ddb66b753ff0-kube-api-access-nwbgt" pod "541835ff-1367-4ed6-8f6e-ddb66b753ff0" (UID: "541835ff-1367-4ed6-8f6e-ddb66b753ff0"). InnerVolumeSpecName "kube-api-access-nwbgt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:01:38.472172 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.467904 4374 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-dshm\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:38.472172 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.467955 4374 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541835ff-1367-4ed6-8f6e-ddb66b753ff0-home\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:38.472172 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.467973 4374 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541835ff-1367-4ed6-8f6e-ddb66b753ff0-tls-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:38.472172 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:38.467991 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwbgt\" (UniqueName: \"kubernetes.io/projected/541835ff-1367-4ed6-8f6e-ddb66b753ff0-kube-api-access-nwbgt\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:39.009160 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:39.009130 4374 generic.go:356] "Generic (PLEG): container finished" podID="541835ff-1367-4ed6-8f6e-ddb66b753ff0" containerID="6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564" exitCode=0 Nov 04 05:01:39.009657 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:39.009180 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" Nov 04 05:01:39.009657 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:39.009220 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" event={"ID":"541835ff-1367-4ed6-8f6e-ddb66b753ff0","Type":"ContainerDied","Data":"6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564"} Nov 04 05:01:39.009657 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:39.009264 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6" event={"ID":"541835ff-1367-4ed6-8f6e-ddb66b753ff0","Type":"ContainerDied","Data":"6ea223fc5fc29d7ac635714adde3013f768def53d64baeee02cbb2f36b818985"} Nov 04 05:01:39.009657 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:39.009281 4374 scope.go:122] "RemoveContainer" containerID="6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564" Nov 04 05:01:39.017927 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:39.017907 4374 scope.go:122] "RemoveContainer" containerID="6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564" Nov 04 05:01:39.018192 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:01:39.018169 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564\": container with ID starting with 6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564 not found: ID does not exist" containerID="6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564" Nov 04 05:01:39.018251 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:39.018200 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564"} err="failed to get container status \"6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564\": rpc error: code = NotFound desc = could not find container \"6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564\": container with ID starting with 6708932521cca22f580321caf6d40615dc2dfc09c64c9af8e16d235b297ff564 not found: ID does not exist" Nov 04 05:01:39.163053 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:39.163023 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6"] Nov 04 05:01:39.184906 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:39.184855 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:39.209367 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:39.209341 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-db8fae/simulator-w6-db8fae-kserve-9768bb699-wdpv6"] Nov 04 05:01:39.915305 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:39.915275 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:40.412525 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:40.412488 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="541835ff-1367-4ed6-8f6e-ddb66b753ff0" path="/var/lib/kubelet/pods/541835ff-1367-4ed6-8f6e-ddb66b753ff0/volumes" Nov 04 05:01:42.145262 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:42.145228 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:42.807162 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:42.807126 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:42.987674 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:42.987644 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:43.777743 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:43.777712 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:43.925274 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:43.925134 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:44.130685 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:44.130601 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:47.216185 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:47.216145 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:47.364807 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:47.364774 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:47.754984 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:47.754887 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:48.032711 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:48.032636 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq"] Nov 04 05:01:48.032918 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:48.032864 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" podUID="751dc7b8-535d-4954-a0db-96f9dad7322c" containerName="maas-api" containerID="cri-o://3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5" gracePeriod=30 Nov 04 05:01:48.139842 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:48.139810 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:48.788101 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:48.788057 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:48.817078 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:48.817049 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 05:01:49.067610 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.067528 4374 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" podUID="751dc7b8-535d-4954-a0db-96f9dad7322c" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.44:8443/health\": dial tcp 10.133.0.44:8443: connect: connection refused" Nov 04 05:01:49.384988 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.384648 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 05:01:49.442499 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.442459 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-metrics-certs\") pod \"751dc7b8-535d-4954-a0db-96f9dad7322c\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " Nov 04 05:01:49.442723 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.442557 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/751dc7b8-535d-4954-a0db-96f9dad7322c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/751dc7b8-535d-4954-a0db-96f9dad7322c-openshift-service-ca\") pod \"751dc7b8-535d-4954-a0db-96f9dad7322c\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " Nov 04 05:01:49.442723 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.442595 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/751dc7b8-535d-4954-a0db-96f9dad7322c-kube-api-access-thtfv\" (UniqueName: \"kubernetes.io/projected/751dc7b8-535d-4954-a0db-96f9dad7322c-kube-api-access-thtfv\") pod \"751dc7b8-535d-4954-a0db-96f9dad7322c\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " Nov 04 05:01:49.442850 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.442746 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-maas-api-tls\") pod \"751dc7b8-535d-4954-a0db-96f9dad7322c\" (UID: \"751dc7b8-535d-4954-a0db-96f9dad7322c\") " Nov 04 05:01:49.444173 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.444138 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/751dc7b8-535d-4954-a0db-96f9dad7322c-openshift-service-ca" pod "751dc7b8-535d-4954-a0db-96f9dad7322c" (UID: "751dc7b8-535d-4954-a0db-96f9dad7322c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:01:49.447737 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.447713 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/751dc7b8-535d-4954-a0db-96f9dad7322c-kube-api-access-thtfv" pod "751dc7b8-535d-4954-a0db-96f9dad7322c" (UID: "751dc7b8-535d-4954-a0db-96f9dad7322c"). InnerVolumeSpecName "kube-api-access-thtfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:01:49.448448 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.448416 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-metrics-certs" pod "751dc7b8-535d-4954-a0db-96f9dad7322c" (UID: "751dc7b8-535d-4954-a0db-96f9dad7322c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:01:49.468450 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.468382 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-maas-api-tls" pod "751dc7b8-535d-4954-a0db-96f9dad7322c" (UID: "751dc7b8-535d-4954-a0db-96f9dad7322c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:01:49.544531 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.544497 4374 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-metrics-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:49.544531 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.544527 4374 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/751dc7b8-535d-4954-a0db-96f9dad7322c-openshift-service-ca\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:49.544531 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.544536 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thtfv\" (UniqueName: \"kubernetes.io/projected/751dc7b8-535d-4954-a0db-96f9dad7322c-kube-api-access-thtfv\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:49.544749 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:49.544546 4374 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/751dc7b8-535d-4954-a0db-96f9dad7322c-maas-api-tls\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:50.034648 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.034582 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:50.076416 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.076369 4374 generic.go:356] "Generic (PLEG): container finished" podID="751dc7b8-535d-4954-a0db-96f9dad7322c" containerID="3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5" exitCode=0 Nov 04 05:01:50.076783 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.076759 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" event={"ID":"751dc7b8-535d-4954-a0db-96f9dad7322c","Type":"ContainerDied","Data":"3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5"} Nov 04 05:01:50.076988 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.076954 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" event={"ID":"751dc7b8-535d-4954-a0db-96f9dad7322c","Type":"ContainerDied","Data":"532df2471b04e12f095d5bee98f4227bc45934488c8fa13c2814e9d43387f444"} Nov 04 05:01:50.077160 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.077145 4374 scope.go:122] "RemoveContainer" containerID="3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5" Nov 04 05:01:50.077401 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.077383 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq" Nov 04 05:01:50.107622 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.107323 4374 scope.go:122] "RemoveContainer" containerID="3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5" Nov 04 05:01:50.108479 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:01:50.107811 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5\": container with ID starting with 3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5 not found: ID does not exist" containerID="3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5" Nov 04 05:01:50.108655 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.107850 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5"} err="failed to get container status \"3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5\": rpc error: code = NotFound desc = could not find container \"3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5\": container with ID starting with 3c4b7c7d7ea60db4ab47f11ef1a40b1b21045ea87a5e59bd5c4debd37c1511c5 not found: ID does not exist" Nov 04 05:01:50.236462 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.236413 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:50.258299 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.258252 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq"] Nov 04 05:01:50.262129 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.262093 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-19bf5b-6cff795759-lz7vq"] Nov 04 05:01:50.494329 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.494296 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="751dc7b8-535d-4954-a0db-96f9dad7322c" path="/var/lib/kubelet/pods/751dc7b8-535d-4954-a0db-96f9dad7322c/volumes" Nov 04 05:01:50.585180 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.585147 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:50.732631 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:50.732594 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:52.371403 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:52.371364 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:54.336213 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.336181 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z"] Nov 04 05:01:54.336662 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.336377 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" podUID="a0075931-f3b3-4932-a18c-f2a172d6677b" containerName="maas-api" containerID="cri-o://250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df" gracePeriod=30 Nov 04 05:01:54.438251 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.438220 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:54.618655 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.618629 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 05:01:54.637740 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.637712 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-metrics-certs\") pod \"a0075931-f3b3-4932-a18c-f2a172d6677b\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " Nov 04 05:01:54.637854 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.637803 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0075931-f3b3-4932-a18c-f2a172d6677b-kube-api-access-89x6h\" (UniqueName: \"kubernetes.io/projected/a0075931-f3b3-4932-a18c-f2a172d6677b-kube-api-access-89x6h\") pod \"a0075931-f3b3-4932-a18c-f2a172d6677b\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " Nov 04 05:01:54.637919 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.637856 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-maas-api-tls\") pod \"a0075931-f3b3-4932-a18c-f2a172d6677b\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " Nov 04 05:01:54.637919 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.637910 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0075931-f3b3-4932-a18c-f2a172d6677b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0075931-f3b3-4932-a18c-f2a172d6677b-openshift-service-ca\") pod \"a0075931-f3b3-4932-a18c-f2a172d6677b\" (UID: \"a0075931-f3b3-4932-a18c-f2a172d6677b\") " Nov 04 05:01:54.638369 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.638311 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0075931-f3b3-4932-a18c-f2a172d6677b-openshift-service-ca" pod "a0075931-f3b3-4932-a18c-f2a172d6677b" (UID: "a0075931-f3b3-4932-a18c-f2a172d6677b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:01:54.639832 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.639811 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-metrics-certs" pod "a0075931-f3b3-4932-a18c-f2a172d6677b" (UID: "a0075931-f3b3-4932-a18c-f2a172d6677b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:01:54.639975 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.639949 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-maas-api-tls" pod "a0075931-f3b3-4932-a18c-f2a172d6677b" (UID: "a0075931-f3b3-4932-a18c-f2a172d6677b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:01:54.639975 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.639965 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0075931-f3b3-4932-a18c-f2a172d6677b-kube-api-access-89x6h" pod "a0075931-f3b3-4932-a18c-f2a172d6677b" (UID: "a0075931-f3b3-4932-a18c-f2a172d6677b"). InnerVolumeSpecName "kube-api-access-89x6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:01:54.739850 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.739735 4374 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-metrics-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:54.739850 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.739778 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89x6h\" (UniqueName: \"kubernetes.io/projected/a0075931-f3b3-4932-a18c-f2a172d6677b-kube-api-access-89x6h\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:54.739850 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.739793 4374 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0075931-f3b3-4932-a18c-f2a172d6677b-maas-api-tls\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:54.739850 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.739807 4374 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0075931-f3b3-4932-a18c-f2a172d6677b-openshift-service-ca\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:54.740807 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:54.740749 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:55.111043 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:55.111010 4374 generic.go:356] "Generic (PLEG): container finished" podID="a0075931-f3b3-4932-a18c-f2a172d6677b" containerID="250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df" exitCode=0 Nov 04 05:01:55.111203 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:55.111056 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" Nov 04 05:01:55.111203 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:55.111079 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" event={"ID":"a0075931-f3b3-4932-a18c-f2a172d6677b","Type":"ContainerDied","Data":"250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df"} Nov 04 05:01:55.111203 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:55.111120 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z" event={"ID":"a0075931-f3b3-4932-a18c-f2a172d6677b","Type":"ContainerDied","Data":"42f1b7eb8b9b8fbfdc55027e0b52fe506b128811513ba30d68cd14696da16f6e"} Nov 04 05:01:55.111203 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:55.111142 4374 scope.go:122] "RemoveContainer" containerID="250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df" Nov 04 05:01:55.120804 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:55.120788 4374 scope.go:122] "RemoveContainer" containerID="250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df" Nov 04 05:01:55.121073 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:01:55.121056 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df\": container with ID starting with 250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df not found: ID does not exist" containerID="250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df" Nov 04 05:01:55.121123 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:55.121081 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df"} err="failed to get container status \"250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df\": rpc error: code = NotFound desc = could not find container \"250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df\": container with ID starting with 250c5f6d5e21ef49a4ce36460e8a23ad650251a0b1316d5de329d321d3dae7df not found: ID does not exist" Nov 04 05:01:55.264306 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:55.264268 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z"] Nov 04 05:01:55.267300 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:55.267263 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:55.270145 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:55.270120 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-e62689-5755f56c7d-qbv5z"] Nov 04 05:01:55.626600 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:55.626566 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:55.789992 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:55.789946 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:55.842953 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:01:55.842916 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 05:01:56.411657 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:56.411607 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0075931-f3b3-4932-a18c-f2a172d6677b" path="/var/lib/kubelet/pods/a0075931-f3b3-4932-a18c-f2a172d6677b/volumes" Nov 04 05:01:56.795257 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:56.795225 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq"] Nov 04 05:01:56.795670 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:56.795423 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" podUID="8fca8cae-8eec-47a3-8efe-7a75bf7113c5" containerName="maas-api" containerID="cri-o://d491c3f604368d4fabe98219170de4dcf5adda666cbb1fbe5443cf0555e6cda4" gracePeriod=30 Nov 04 05:01:56.898860 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:56.898824 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:57.295171 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:57.295122 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:57.442146 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:57.442116 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:57.813970 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:57.813940 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp"] Nov 04 05:01:57.814376 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:57.814152 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerName="istio-proxy" containerID="cri-o://7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada" gracePeriod=30 Nov 04 05:01:57.913003 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:57.912969 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:58.130944 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.130914 4374 generic.go:356] "Generic (PLEG): container finished" podID="8fca8cae-8eec-47a3-8efe-7a75bf7113c5" containerID="d491c3f604368d4fabe98219170de4dcf5adda666cbb1fbe5443cf0555e6cda4" exitCode=0 Nov 04 05:01:58.131083 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.130986 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" event={"ID":"8fca8cae-8eec-47a3-8efe-7a75bf7113c5","Type":"ContainerDied","Data":"d491c3f604368d4fabe98219170de4dcf5adda666cbb1fbe5443cf0555e6cda4"} Nov 04 05:01:58.131083 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.131036 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" event={"ID":"8fca8cae-8eec-47a3-8efe-7a75bf7113c5","Type":"ContainerDied","Data":"05c457f44772f1d1e5f6a5966c548cbccf5c2bfe00fdd7a857df9c6c6c9a5c59"} Nov 04 05:01:58.131083 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.131056 4374 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05c457f44772f1d1e5f6a5966c548cbccf5c2bfe00fdd7a857df9c6c6c9a5c59" Nov 04 05:01:58.135447 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.135427 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 05:01:58.198952 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.198925 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-maas-api-tls\") pod \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " Nov 04 05:01:58.199105 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.198975 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-openshift-service-ca\") pod \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " Nov 04 05:01:58.199105 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.198999 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-kube-api-access-6djx8\" (UniqueName: \"kubernetes.io/projected/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-kube-api-access-6djx8\") pod \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " Nov 04 05:01:58.199401 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.199375 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-openshift-service-ca" pod "8fca8cae-8eec-47a3-8efe-7a75bf7113c5" (UID: "8fca8cae-8eec-47a3-8efe-7a75bf7113c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:01:58.201047 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.201020 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-maas-api-tls" pod "8fca8cae-8eec-47a3-8efe-7a75bf7113c5" (UID: "8fca8cae-8eec-47a3-8efe-7a75bf7113c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:01:58.201136 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.201067 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-kube-api-access-6djx8" pod "8fca8cae-8eec-47a3-8efe-7a75bf7113c5" (UID: "8fca8cae-8eec-47a3-8efe-7a75bf7113c5"). InnerVolumeSpecName "kube-api-access-6djx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:01:58.300241 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.300209 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-metrics-certs\") pod \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\" (UID: \"8fca8cae-8eec-47a3-8efe-7a75bf7113c5\") " Nov 04 05:01:58.300522 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.300508 4374 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-maas-api-tls\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:58.300578 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.300526 4374 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-openshift-service-ca\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:58.300578 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.300536 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6djx8\" (UniqueName: \"kubernetes.io/projected/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-kube-api-access-6djx8\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:58.302405 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.302376 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-metrics-certs" pod "8fca8cae-8eec-47a3-8efe-7a75bf7113c5" (UID: "8fca8cae-8eec-47a3-8efe-7a75bf7113c5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:01:58.401514 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.401440 4374 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fca8cae-8eec-47a3-8efe-7a75bf7113c5-metrics-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:01:58.931600 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:58.931570 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:59.079565 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:59.079511 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:59.140205 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:59.140104 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq" Nov 04 05:01:59.232237 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:59.232203 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:01:59.296226 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:59.296193 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq"] Nov 04 05:01:59.342294 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:01:59.342269 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-db8fae-7646bb77cf-6jncq"] Nov 04 05:02:00.015450 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:00.015408 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:00.155912 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:00.155880 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:00.388221 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:00.388144 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:00.414305 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:00.414275 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8fca8cae-8eec-47a3-8efe-7a75bf7113c5" path="/var/lib/kubelet/pods/8fca8cae-8eec-47a3-8efe-7a75bf7113c5/volumes" Nov 04 05:02:01.221471 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:01.220397 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:01.988364 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:01.988330 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:02.130450 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:02.130420 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:03.074087 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.074062 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 05:02:03.152744 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.152659 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-data\") pod \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " Nov 04 05:02:03.152744 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.152714 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istiod-ca-cert\") pod \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " Nov 04 05:02:03.152976 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.152767 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-podinfo\") pod \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " Nov 04 05:02:03.152976 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.152808 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-envoy\") pod \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " Nov 04 05:02:03.152976 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.152934 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-socket\") pod \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " Nov 04 05:02:03.153204 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.152988 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-data" pod "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" (UID: "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:03.153204 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153025 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-certs\") pod \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " Nov 04 05:02:03.153204 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153079 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-token\" (UniqueName: \"kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-token\") pod \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " Nov 04 05:02:03.153204 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153153 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-credential-socket\") pod \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " Nov 04 05:02:03.153204 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153182 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-kube-api-access-f8hsr\" (UniqueName: \"kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-kube-api-access-f8hsr\") pod \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\" (UID: \"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b\") " Nov 04 05:02:03.153445 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153251 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-socket" pod "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" (UID: "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:03.153445 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153295 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istiod-ca-cert" pod "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" (UID: "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:02:03.153445 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153358 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-certs" pod "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" (UID: "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:03.153607 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153468 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-credential-socket" pod "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" (UID: "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:03.153735 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153716 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:03.153830 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153742 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:03.153830 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153759 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:03.153830 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153773 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:03.153830 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.153786 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:03.155336 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.155311 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-token" pod "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" (UID: "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:03.155440 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.155398 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-kube-api-access-f8hsr" pod "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" (UID: "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b"). InnerVolumeSpecName "kube-api-access-f8hsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:03.155518 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.155497 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-podinfo" pod "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" (UID: "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:02:03.155585 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.155568 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-envoy" pod "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" (UID: "5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:03.161816 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.161792 4374 generic.go:356] "Generic (PLEG): container finished" podID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerID="7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada" exitCode=0 Nov 04 05:02:03.161952 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.161839 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" event={"ID":"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b","Type":"ContainerDied","Data":"7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada"} Nov 04 05:02:03.161952 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.161854 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" Nov 04 05:02:03.162051 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.161925 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp" event={"ID":"5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b","Type":"ContainerDied","Data":"3e02865e99a7992533803c314159714eb74360aab493de8cc94ea4dd715aaa47"} Nov 04 05:02:03.162090 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.161939 4374 scope.go:122] "RemoveContainer" containerID="7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada" Nov 04 05:02:03.180455 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.180438 4374 scope.go:122] "RemoveContainer" containerID="90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1" Nov 04 05:02:03.188638 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.188616 4374 scope.go:122] "RemoveContainer" containerID="7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada" Nov 04 05:02:03.188924 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:02:03.188900 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada\": container with ID starting with 7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada not found: ID does not exist" containerID="7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada" Nov 04 05:02:03.189049 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.188934 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada"} err="failed to get container status \"7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada\": rpc error: code = NotFound desc = could not find container \"7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada\": container with ID starting with 7422cb4919737ec135305367a9ae5740eedaff7a2c9950b4936c9ca4ea7b3ada not found: ID does not exist" Nov 04 05:02:03.189049 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.188956 4374 scope.go:122] "RemoveContainer" containerID="90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1" Nov 04 05:02:03.189315 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:02:03.189295 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1\": container with ID starting with 90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1 not found: ID does not exist" containerID="90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1" Nov 04 05:02:03.189400 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.189324 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1"} err="failed to get container status \"90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1\": rpc error: code = NotFound desc = could not find container \"90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1\": container with ID starting with 90df8bb2fccac7fad36ab81dc376e419526c8593fcb348e3d64f4dd3abd8eee1 not found: ID does not exist" Nov 04 05:02:03.254466 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.254435 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:03.254466 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.254461 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8hsr\" (UniqueName: \"kubernetes.io/projected/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-kube-api-access-f8hsr\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:03.254643 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.254478 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:03.254643 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.254491 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:03.293270 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.293236 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp"] Nov 04 05:02:03.297827 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.297799 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-19bf5b-openshift-default-5c54fdccbf-kg8qp"] Nov 04 05:02:03.635373 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:03.635339 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:04.154485 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:04.154454 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87"] Nov 04 05:02:04.154885 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:04.154648 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerName="istio-proxy" containerID="cri-o://fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597" gracePeriod=30 Nov 04 05:02:04.158569 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:04.158545 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:04.335316 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:04.335283 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:04.463267 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:04.463184 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" path="/var/lib/kubelet/pods/5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b/volumes" Nov 04 05:02:04.904840 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:04.904812 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:05.050301 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:05.050257 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:05.513066 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:05.513030 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:05.661161 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:05.661124 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:06.496314 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:06.496278 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:06.638364 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:06.638336 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:07.020046 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:07.020018 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:07.022360 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:07.022339 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk"] Nov 04 05:02:07.022563 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:07.022542 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" podUID="418c3a14-fbcb-4566-9009-5d4799d59593" containerName="istio-proxy" containerID="cri-o://524dc4def0b0202937f2810d18ff711396491448d891302bb523dcfe31134d37" gracePeriod=30 Nov 04 05:02:07.369008 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:07.368827 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:08.012547 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:08.012518 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:08.556280 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:08.556251 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr"] Nov 04 05:02:08.556504 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:08.556478 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" podUID="f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" containerName="istio-proxy" containerID="cri-o://1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863" gracePeriod=30 Nov 04 05:02:08.565130 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:08.565099 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:09.271816 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.271790 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:09.436644 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.436622 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 05:02:09.540058 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540028 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-socket\") pod \"9431eac2-29af-4785-8277-ab9bed9fd12d\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " Nov 04 05:02:09.540237 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540077 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-credential-socket\") pod \"9431eac2-29af-4785-8277-ab9bed9fd12d\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " Nov 04 05:02:09.540237 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540100 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9431eac2-29af-4785-8277-ab9bed9fd12d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9431eac2-29af-4785-8277-ab9bed9fd12d-istiod-ca-cert\") pod \"9431eac2-29af-4785-8277-ab9bed9fd12d\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " Nov 04 05:02:09.540237 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540166 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-podinfo\") pod \"9431eac2-29af-4785-8277-ab9bed9fd12d\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " Nov 04 05:02:09.540237 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540204 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-kube-api-access-52887\" (UniqueName: \"kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-kube-api-access-52887\") pod \"9431eac2-29af-4785-8277-ab9bed9fd12d\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " Nov 04 05:02:09.540480 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540267 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-certs\") pod \"9431eac2-29af-4785-8277-ab9bed9fd12d\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " Nov 04 05:02:09.540480 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540297 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-token\" (UniqueName: \"kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-token\") pod \"9431eac2-29af-4785-8277-ab9bed9fd12d\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " Nov 04 05:02:09.540480 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540327 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-credential-socket" pod "9431eac2-29af-4785-8277-ab9bed9fd12d" (UID: "9431eac2-29af-4785-8277-ab9bed9fd12d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:09.540480 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540387 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-data\") pod \"9431eac2-29af-4785-8277-ab9bed9fd12d\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " Nov 04 05:02:09.540480 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540464 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-envoy\") pod \"9431eac2-29af-4785-8277-ab9bed9fd12d\" (UID: \"9431eac2-29af-4785-8277-ab9bed9fd12d\") " Nov 04 05:02:09.540735 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540385 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-socket" pod "9431eac2-29af-4785-8277-ab9bed9fd12d" (UID: "9431eac2-29af-4785-8277-ab9bed9fd12d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:09.540735 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540514 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-certs" pod "9431eac2-29af-4785-8277-ab9bed9fd12d" (UID: "9431eac2-29af-4785-8277-ab9bed9fd12d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:09.540735 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540615 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-data" pod "9431eac2-29af-4785-8277-ab9bed9fd12d" (UID: "9431eac2-29af-4785-8277-ab9bed9fd12d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:09.540915 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.540885 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9431eac2-29af-4785-8277-ab9bed9fd12d-istiod-ca-cert" pod "9431eac2-29af-4785-8277-ab9bed9fd12d" (UID: "9431eac2-29af-4785-8277-ab9bed9fd12d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:02:09.541130 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.541049 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:09.541130 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.541077 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:09.541130 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.541090 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:09.541403 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.541359 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:09.541403 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.541384 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9431eac2-29af-4785-8277-ab9bed9fd12d-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:09.575818 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.567010 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-podinfo" pod "9431eac2-29af-4785-8277-ab9bed9fd12d" (UID: "9431eac2-29af-4785-8277-ab9bed9fd12d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:02:09.575818 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.567113 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-envoy" pod "9431eac2-29af-4785-8277-ab9bed9fd12d" (UID: "9431eac2-29af-4785-8277-ab9bed9fd12d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:09.575818 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.567178 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-token" pod "9431eac2-29af-4785-8277-ab9bed9fd12d" (UID: "9431eac2-29af-4785-8277-ab9bed9fd12d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:09.575818 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.567231 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-kube-api-access-52887" pod "9431eac2-29af-4785-8277-ab9bed9fd12d" (UID: "9431eac2-29af-4785-8277-ab9bed9fd12d"). InnerVolumeSpecName "kube-api-access-52887". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:09.642799 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.642762 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:09.642799 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.642792 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52887\" (UniqueName: \"kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-kube-api-access-52887\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:09.642799 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.642802 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:09.643075 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.642811 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9431eac2-29af-4785-8277-ab9bed9fd12d-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:09.736088 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.736059 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:09.994348 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.993648 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb"] Nov 04 05:02:09.994348 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:09.994000 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" podUID="759be065-f27c-4886-8deb-1d56ced9139f" containerName="istio-proxy" containerID="cri-o://057e5aeaa6a9004a02a8b161726dc2dfe6410090839437756b275499a85d53c4" gracePeriod=30 Nov 04 05:02:10.203088 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.203054 4374 generic.go:356] "Generic (PLEG): container finished" podID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerID="fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597" exitCode=0 Nov 04 05:02:10.203254 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.203118 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" Nov 04 05:02:10.203254 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.203130 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" event={"ID":"9431eac2-29af-4785-8277-ab9bed9fd12d","Type":"ContainerDied","Data":"fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597"} Nov 04 05:02:10.203254 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.203185 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87" event={"ID":"9431eac2-29af-4785-8277-ab9bed9fd12d","Type":"ContainerDied","Data":"2b6c93f5186b8eb9789bd61fc2bcbc64443ce095525d7cdc920899a738fc6033"} Nov 04 05:02:10.203254 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.203207 4374 scope.go:122] "RemoveContainer" containerID="fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597" Nov 04 05:02:10.212787 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.212772 4374 scope.go:122] "RemoveContainer" containerID="314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717" Nov 04 05:02:10.220805 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.220787 4374 scope.go:122] "RemoveContainer" containerID="fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597" Nov 04 05:02:10.221120 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:02:10.221100 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597\": container with ID starting with fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597 not found: ID does not exist" containerID="fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597" Nov 04 05:02:10.221194 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.221129 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597"} err="failed to get container status \"fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597\": rpc error: code = NotFound desc = could not find container \"fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597\": container with ID starting with fa5d8964af5ff01713ecbfdbf1160feab1596ca3e9c37efa25af10eb407e0597 not found: ID does not exist" Nov 04 05:02:10.221194 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.221149 4374 scope.go:122] "RemoveContainer" containerID="314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717" Nov 04 05:02:10.221331 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:02:10.221316 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717\": container with ID starting with 314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717 not found: ID does not exist" containerID="314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717" Nov 04 05:02:10.221370 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.221335 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717"} err="failed to get container status \"314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717\": rpc error: code = NotFound desc = could not find container \"314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717\": container with ID starting with 314c4631f883c6c4c58f41ba72f70bb1fd542f5f3e51c7f99c67b63314c13717 not found: ID does not exist" Nov 04 05:02:10.359017 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.358921 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87"] Nov 04 05:02:10.364970 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.364943 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-204fa0-openshift-default-5cb9795595-2ps87"] Nov 04 05:02:10.411010 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.410979 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" path="/var/lib/kubelet/pods/9431eac2-29af-4785-8277-ab9bed9fd12d/volumes" Nov 04 05:02:10.682064 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.681976 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:10.828114 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:10.828078 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:11.755793 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.755762 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp"] Nov 04 05:02:11.756254 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756239 4374 cpu_manager.go:414] "containerMap: removing container" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerName="istio-proxy" Nov 04 05:02:11.756254 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756255 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerName="istio-proxy" Nov 04 05:02:11.756333 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756271 4374 cpu_manager.go:414] "containerMap: removing container" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerName="istio-proxy" Nov 04 05:02:11.756333 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756277 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerName="istio-proxy" Nov 04 05:02:11.756333 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756286 4374 cpu_manager.go:414] "containerMap: removing container" podUID="a0075931-f3b3-4932-a18c-f2a172d6677b" containerName="maas-api" Nov 04 05:02:11.756333 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756291 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0075931-f3b3-4932-a18c-f2a172d6677b" containerName="maas-api" Nov 04 05:02:11.756333 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756299 4374 cpu_manager.go:414] "containerMap: removing container" podUID="8fca8cae-8eec-47a3-8efe-7a75bf7113c5" containerName="maas-api" Nov 04 05:02:11.756333 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756304 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8fca8cae-8eec-47a3-8efe-7a75bf7113c5" containerName="maas-api" Nov 04 05:02:11.756333 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756314 4374 cpu_manager.go:414] "containerMap: removing container" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerName="istio-proxy" Nov 04 05:02:11.756333 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756318 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerName="istio-proxy" Nov 04 05:02:11.756333 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756329 4374 cpu_manager.go:414] "containerMap: removing container" podUID="751dc7b8-535d-4954-a0db-96f9dad7322c" containerName="maas-api" Nov 04 05:02:11.756333 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756334 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="751dc7b8-535d-4954-a0db-96f9dad7322c" containerName="maas-api" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756340 4374 cpu_manager.go:414] "containerMap: removing container" podUID="541835ff-1367-4ed6-8f6e-ddb66b753ff0" containerName="main" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756344 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="541835ff-1367-4ed6-8f6e-ddb66b753ff0" containerName="main" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756373 4374 cpu_manager.go:414] "containerMap: removing container" podUID="e25b779b-ecae-46cf-830a-4f5ededa9008" containerName="main" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756378 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e25b779b-ecae-46cf-830a-4f5ededa9008" containerName="main" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756448 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerName="istio-proxy" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756458 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerName="istio-proxy" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756465 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0075931-f3b3-4932-a18c-f2a172d6677b" containerName="maas-api" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756471 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="8fca8cae-8eec-47a3-8efe-7a75bf7113c5" containerName="maas-api" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756477 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerName="istio-proxy" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756482 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="9431eac2-29af-4785-8277-ab9bed9fd12d" containerName="istio-proxy" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756487 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="541835ff-1367-4ed6-8f6e-ddb66b753ff0" containerName="main" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756493 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="751dc7b8-535d-4954-a0db-96f9dad7322c" containerName="maas-api" Nov 04 05:02:11.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.756499 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="e25b779b-ecae-46cf-830a-4f5ededa9008" containerName="main" Nov 04 05:02:11.760931 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.760910 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.866312 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.866275 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-credential-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.866476 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.866327 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-podinfo\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.866476 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.866411 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-token\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.866476 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.866451 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-envoy\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.866631 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.866532 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f576ad1-48cd-469e-9c44-6a42322968e7-istiod-ca-cert\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.866631 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.866556 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpgbl\" (UniqueName: \"kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-kube-api-access-qpgbl\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.866631 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.866621 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-certs\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.866727 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.866651 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-data\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.866727 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.866720 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.892858 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.892825 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp"] Nov 04 05:02:11.967966 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.967933 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f576ad1-48cd-469e-9c44-6a42322968e7-istiod-ca-cert\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968103 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.967975 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgbl\" (UniqueName: \"kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-kube-api-access-qpgbl\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968103 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.968025 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-certs\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968103 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.968066 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-data\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968242 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.968115 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968242 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.968201 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-credential-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968341 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.968263 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-podinfo\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968341 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.968306 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-token\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968459 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.968352 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-envoy\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968496 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.968459 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-certs\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968536 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.968493 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-data\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968710 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.968688 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f576ad1-48cd-469e-9c44-6a42322968e7-istiod-ca-cert\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968809 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.968757 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-credential-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.968809 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.968787 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.970539 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.970521 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-envoy\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:11.970679 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:11.970662 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-podinfo\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:12.035514 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.035445 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-token\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:12.037275 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.037250 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpgbl\" (UniqueName: \"kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-kube-api-access-qpgbl\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:12.073349 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.073327 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:12.215644 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.215610 4374 generic.go:356] "Generic (PLEG): container finished" podID="418c3a14-fbcb-4566-9009-5d4799d59593" containerID="524dc4def0b0202937f2810d18ff711396491448d891302bb523dcfe31134d37" exitCode=0 Nov 04 05:02:12.215812 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.215694 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" event={"ID":"418c3a14-fbcb-4566-9009-5d4799d59593","Type":"ContainerDied","Data":"524dc4def0b0202937f2810d18ff711396491448d891302bb523dcfe31134d37"} Nov 04 05:02:12.269255 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.269230 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 05:02:12.373222 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.373138 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-credential-socket\") pod \"418c3a14-fbcb-4566-9009-5d4799d59593\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " Nov 04 05:02:12.373361 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.373279 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-istio-token\" (UniqueName: \"kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-istio-token\") pod \"418c3a14-fbcb-4566-9009-5d4799d59593\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " Nov 04 05:02:12.373361 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.373313 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-kube-api-access-zt7dr\" (UniqueName: \"kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-kube-api-access-zt7dr\") pod \"418c3a14-fbcb-4566-9009-5d4799d59593\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " Nov 04 05:02:12.373361 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.373336 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-envoy\") pod \"418c3a14-fbcb-4566-9009-5d4799d59593\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " Nov 04 05:02:12.373361 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.373355 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-socket\") pod \"418c3a14-fbcb-4566-9009-5d4799d59593\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " Nov 04 05:02:12.373584 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.373405 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-credential-socket" pod "418c3a14-fbcb-4566-9009-5d4799d59593" (UID: "418c3a14-fbcb-4566-9009-5d4799d59593"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:12.373584 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.373455 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/418c3a14-fbcb-4566-9009-5d4799d59593-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/418c3a14-fbcb-4566-9009-5d4799d59593-istio-podinfo\") pod \"418c3a14-fbcb-4566-9009-5d4799d59593\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " Nov 04 05:02:12.373584 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.373489 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-certs\") pod \"418c3a14-fbcb-4566-9009-5d4799d59593\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " Nov 04 05:02:12.373745 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.373582 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/418c3a14-fbcb-4566-9009-5d4799d59593-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/418c3a14-fbcb-4566-9009-5d4799d59593-istiod-ca-cert\") pod \"418c3a14-fbcb-4566-9009-5d4799d59593\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " Nov 04 05:02:12.373745 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.373640 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-data\") pod \"418c3a14-fbcb-4566-9009-5d4799d59593\" (UID: \"418c3a14-fbcb-4566-9009-5d4799d59593\") " Nov 04 05:02:12.373745 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.373659 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-socket" pod "418c3a14-fbcb-4566-9009-5d4799d59593" (UID: "418c3a14-fbcb-4566-9009-5d4799d59593"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:12.373940 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.373844 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-certs" pod "418c3a14-fbcb-4566-9009-5d4799d59593" (UID: "418c3a14-fbcb-4566-9009-5d4799d59593"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:12.374069 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.374035 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-data" pod "418c3a14-fbcb-4566-9009-5d4799d59593" (UID: "418c3a14-fbcb-4566-9009-5d4799d59593"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:12.374192 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.374110 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/418c3a14-fbcb-4566-9009-5d4799d59593-istiod-ca-cert" pod "418c3a14-fbcb-4566-9009-5d4799d59593" (UID: "418c3a14-fbcb-4566-9009-5d4799d59593"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:02:12.374241 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.374216 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:12.374241 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.374235 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:12.374331 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.374250 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/418c3a14-fbcb-4566-9009-5d4799d59593-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:12.374331 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.374266 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:12.374331 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.374281 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:12.375622 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.375583 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-envoy" pod "418c3a14-fbcb-4566-9009-5d4799d59593" (UID: "418c3a14-fbcb-4566-9009-5d4799d59593"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:12.375804 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.375786 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-kube-api-access-zt7dr" pod "418c3a14-fbcb-4566-9009-5d4799d59593" (UID: "418c3a14-fbcb-4566-9009-5d4799d59593"). InnerVolumeSpecName "kube-api-access-zt7dr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:12.376083 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.376062 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/418c3a14-fbcb-4566-9009-5d4799d59593-istio-podinfo" pod "418c3a14-fbcb-4566-9009-5d4799d59593" (UID: "418c3a14-fbcb-4566-9009-5d4799d59593"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:02:12.376141 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.376080 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-istio-token" pod "418c3a14-fbcb-4566-9009-5d4799d59593" (UID: "418c3a14-fbcb-4566-9009-5d4799d59593"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:12.475061 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.475034 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:12.475061 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.475061 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt7dr\" (UniqueName: \"kubernetes.io/projected/418c3a14-fbcb-4566-9009-5d4799d59593-kube-api-access-zt7dr\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:12.475233 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.475072 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/418c3a14-fbcb-4566-9009-5d4799d59593-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:12.475233 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.475080 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/418c3a14-fbcb-4566-9009-5d4799d59593-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:12.533467 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.533442 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp"] Nov 04 05:02:12.537339 ip-10-0-140-242 kubenswrapper[4374]: W1104 05:02:12.537083 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f576ad1_48cd_469e_9c44_6a42322968e7.slice/crio-c09c50bbf88dd64cafe3f30e583f8265dfdd63cabd7811881ff4053de824e0fb WatchSource:0}: Error finding container c09c50bbf88dd64cafe3f30e583f8265dfdd63cabd7811881ff4053de824e0fb: Status 404 returned error can't find the container with id c09c50bbf88dd64cafe3f30e583f8265dfdd63cabd7811881ff4053de824e0fb Nov 04 05:02:12.540461 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.540414 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:02:12.540577 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.540509 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:02:12.540577 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.540551 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:02:12.706862 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.706761 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:12.852995 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:12.852961 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:13.086632 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.086599 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:13.230966 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.230849 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:13.234257 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.234225 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" event={"ID":"418c3a14-fbcb-4566-9009-5d4799d59593","Type":"ContainerDied","Data":"b7b0f3478f7e7e1ae24dfbf0f82787fae67aedf12c9676508de6957caa0f130d"} Nov 04 05:02:13.234379 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.234241 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk" Nov 04 05:02:13.234379 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.234361 4374 scope.go:122] "RemoveContainer" containerID="524dc4def0b0202937f2810d18ff711396491448d891302bb523dcfe31134d37" Nov 04 05:02:13.235943 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.235912 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" event={"ID":"7f576ad1-48cd-469e-9c44-6a42322968e7","Type":"ContainerStarted","Data":"1ac2283a140ca82af1f52c4f08770864c6dc855f55985975fcfeddc2e77204c9"} Nov 04 05:02:13.236053 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.235958 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" event={"ID":"7f576ad1-48cd-469e-9c44-6a42322968e7","Type":"ContainerStarted","Data":"c09c50bbf88dd64cafe3f30e583f8265dfdd63cabd7811881ff4053de824e0fb"} Nov 04 05:02:13.389668 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.389640 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk"] Nov 04 05:02:13.436288 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.436255 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-e62689-openshift-default-b5d66459-zbtjk"] Nov 04 05:02:13.502611 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.502518 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" podStartSLOduration=2.502504133 podStartE2EDuration="2.502504133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:02:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:02:13.501163571 +0000 UTC m=+975.212854711" watchObservedRunningTime="2026-11-04 05:02:13.502504133 +0000 UTC m=+975.214195271" Nov 04 05:02:13.598728 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.598698 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:13.872890 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.871066 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:02:13.994066 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.993464 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-token\" (UniqueName: \"kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-token\") pod \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " Nov 04 05:02:13.994066 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.993572 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-credential-socket\") pod \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " Nov 04 05:02:13.994066 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.993620 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-podinfo\") pod \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " Nov 04 05:02:13.994066 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.993702 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-socket\") pod \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " Nov 04 05:02:13.994066 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.993765 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-kube-api-access-qmk9h\" (UniqueName: \"kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-kube-api-access-qmk9h\") pod \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " Nov 04 05:02:13.994066 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.993793 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-certs\") pod \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " Nov 04 05:02:13.994066 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.993830 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-data\") pod \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " Nov 04 05:02:13.994066 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.993914 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-envoy\") pod \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " Nov 04 05:02:13.994066 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.993946 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istiod-ca-cert\") pod \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\" (UID: \"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da\") " Nov 04 05:02:13.994655 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.994594 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-data" pod "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" (UID: "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:13.994782 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.994751 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-certs" pod "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" (UID: "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:13.994988 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.994967 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-socket" pod "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" (UID: "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:13.995078 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.994969 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istiod-ca-cert" pod "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" (UID: "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:02:13.995261 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.995227 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-credential-socket" pod "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" (UID: "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:13.996944 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.996907 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-envoy" pod "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" (UID: "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:13.997296 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.997276 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-kube-api-access-qmk9h" pod "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" (UID: "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da"). InnerVolumeSpecName "kube-api-access-qmk9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:13.999245 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:13.999224 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-token" pod "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" (UID: "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:14.001920 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.001892 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-podinfo" pod "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" (UID: "f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:02:14.075337 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.075247 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:14.077457 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.076691 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 05:02:14.077457 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.076831 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" podUID="7f576ad1-48cd-469e-9c44-6a42322968e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 05:02:14.096210 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.096119 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmk9h\" (UniqueName: \"kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-kube-api-access-qmk9h\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:14.096210 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.096153 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:14.096210 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.096163 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:14.096210 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.096172 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:14.096210 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.096181 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:14.096210 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.096189 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:14.096210 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.096198 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:14.096210 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.096207 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:14.096210 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.096217 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:14.243674 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.243635 4374 generic.go:356] "Generic (PLEG): container finished" podID="f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" containerID="1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863" exitCode=0 Nov 04 05:02:14.243837 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.243775 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" Nov 04 05:02:14.243837 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.243807 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" event={"ID":"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da","Type":"ContainerDied","Data":"1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863"} Nov 04 05:02:14.243948 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.243851 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr" event={"ID":"f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da","Type":"ContainerDied","Data":"6df3bebbbc375c89561d30c02dd92a8caea199a39ad5153f7228ac9cbc061c82"} Nov 04 05:02:14.243948 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.243897 4374 scope.go:122] "RemoveContainer" containerID="1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863" Nov 04 05:02:14.253328 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.253311 4374 scope.go:122] "RemoveContainer" containerID="1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863" Nov 04 05:02:14.253559 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:02:14.253541 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863\": container with ID starting with 1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863 not found: ID does not exist" containerID="1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863" Nov 04 05:02:14.253613 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.253571 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863"} err="failed to get container status \"1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863\": rpc error: code = NotFound desc = could not find container \"1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863\": container with ID starting with 1aec6b47ee87e508769309882e1607e48699a80a5c28cffaa79def3054a63863 not found: ID does not exist" Nov 04 05:02:14.398068 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.397986 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr"] Nov 04 05:02:14.401659 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.401633 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-db8fae-openshift-default-55bbb8f84f-qlzlr"] Nov 04 05:02:14.432732 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.432704 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="418c3a14-fbcb-4566-9009-5d4799d59593" path="/var/lib/kubelet/pods/418c3a14-fbcb-4566-9009-5d4799d59593/volumes" Nov 04 05:02:14.433153 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.433138 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" path="/var/lib/kubelet/pods/f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da/volumes" Nov 04 05:02:14.476700 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:14.476663 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:15.074058 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.074026 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 05:02:15.074385 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.074083 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" podUID="7f576ad1-48cd-469e-9c44-6a42322968e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 05:02:15.232898 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.231505 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:15.254451 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.254315 4374 generic.go:356] "Generic (PLEG): container finished" podID="759be065-f27c-4886-8deb-1d56ced9139f" containerID="057e5aeaa6a9004a02a8b161726dc2dfe6410090839437756b275499a85d53c4" exitCode=0 Nov 04 05:02:15.254451 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.254350 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" event={"ID":"759be065-f27c-4886-8deb-1d56ced9139f","Type":"ContainerDied","Data":"057e5aeaa6a9004a02a8b161726dc2dfe6410090839437756b275499a85d53c4"} Nov 04 05:02:15.330889 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.324229 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 05:02:15.440201 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.440167 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/759be065-f27c-4886-8deb-1d56ced9139f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/759be065-f27c-4886-8deb-1d56ced9139f-istio-podinfo\") pod \"759be065-f27c-4886-8deb-1d56ced9139f\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " Nov 04 05:02:15.440385 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.440218 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-envoy\") pod \"759be065-f27c-4886-8deb-1d56ced9139f\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " Nov 04 05:02:15.440385 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.440292 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-socket\") pod \"759be065-f27c-4886-8deb-1d56ced9139f\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " Nov 04 05:02:15.440385 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.440331 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-istio-token\" (UniqueName: \"kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-istio-token\") pod \"759be065-f27c-4886-8deb-1d56ced9139f\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " Nov 04 05:02:15.440385 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.440360 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-credential-socket\") pod \"759be065-f27c-4886-8deb-1d56ced9139f\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " Nov 04 05:02:15.440624 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.440403 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-kube-api-access-rqcjn\" (UniqueName: \"kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-kube-api-access-rqcjn\") pod \"759be065-f27c-4886-8deb-1d56ced9139f\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " Nov 04 05:02:15.440624 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.440485 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-certs\") pod \"759be065-f27c-4886-8deb-1d56ced9139f\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " Nov 04 05:02:15.440624 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.440521 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/759be065-f27c-4886-8deb-1d56ced9139f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/759be065-f27c-4886-8deb-1d56ced9139f-istiod-ca-cert\") pod \"759be065-f27c-4886-8deb-1d56ced9139f\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " Nov 04 05:02:15.440624 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.440568 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-data\") pod \"759be065-f27c-4886-8deb-1d56ced9139f\" (UID: \"759be065-f27c-4886-8deb-1d56ced9139f\") " Nov 04 05:02:15.440818 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.440717 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-socket" pod "759be065-f27c-4886-8deb-1d56ced9139f" (UID: "759be065-f27c-4886-8deb-1d56ced9139f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:15.441162 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.441098 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:15.442028 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.441885 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/759be065-f27c-4886-8deb-1d56ced9139f-istiod-ca-cert" pod "759be065-f27c-4886-8deb-1d56ced9139f" (UID: "759be065-f27c-4886-8deb-1d56ced9139f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:02:15.442028 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.441957 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-certs" pod "759be065-f27c-4886-8deb-1d56ced9139f" (UID: "759be065-f27c-4886-8deb-1d56ced9139f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:15.442028 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.442007 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-credential-socket" pod "759be065-f27c-4886-8deb-1d56ced9139f" (UID: "759be065-f27c-4886-8deb-1d56ced9139f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:15.442336 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.442165 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-data" pod "759be065-f27c-4886-8deb-1d56ced9139f" (UID: "759be065-f27c-4886-8deb-1d56ced9139f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:15.444970 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.444700 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/759be065-f27c-4886-8deb-1d56ced9139f-istio-podinfo" pod "759be065-f27c-4886-8deb-1d56ced9139f" (UID: "759be065-f27c-4886-8deb-1d56ced9139f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:02:15.445290 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.445256 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-kube-api-access-rqcjn" pod "759be065-f27c-4886-8deb-1d56ced9139f" (UID: "759be065-f27c-4886-8deb-1d56ced9139f"). InnerVolumeSpecName "kube-api-access-rqcjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:15.445805 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.445771 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-istio-token" pod "759be065-f27c-4886-8deb-1d56ced9139f" (UID: "759be065-f27c-4886-8deb-1d56ced9139f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:15.446271 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.446249 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-envoy" pod "759be065-f27c-4886-8deb-1d56ced9139f" (UID: "759be065-f27c-4886-8deb-1d56ced9139f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:15.542081 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.541989 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:15.542081 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.542020 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqcjn\" (UniqueName: \"kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-kube-api-access-rqcjn\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:15.542081 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.542032 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:15.542081 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.542041 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/759be065-f27c-4886-8deb-1d56ced9139f-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:15.542081 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.542050 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:15.542081 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.542061 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/759be065-f27c-4886-8deb-1d56ced9139f-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:15.542081 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.542075 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/759be065-f27c-4886-8deb-1d56ced9139f-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:15.542081 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:15.542087 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/759be065-f27c-4886-8deb-1d56ced9139f-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:16.074305 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:16.073956 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 05:02:16.074305 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:16.074024 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" podUID="7f576ad1-48cd-469e-9c44-6a42322968e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 05:02:16.260287 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:16.260246 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" event={"ID":"759be065-f27c-4886-8deb-1d56ced9139f","Type":"ContainerDied","Data":"c332d48cf0f321cda1fb467a2eead53c62f833762aa0b1d122b27303915eee9e"} Nov 04 05:02:16.260439 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:16.260306 4374 scope.go:122] "RemoveContainer" containerID="057e5aeaa6a9004a02a8b161726dc2dfe6410090839437756b275499a85d53c4" Nov 04 05:02:16.260439 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:16.260260 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb" Nov 04 05:02:16.428972 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:16.425623 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb"] Nov 04 05:02:16.471742 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:16.471710 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4t5gb"] Nov 04 05:02:17.074415 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:17.074378 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 05:02:17.074793 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:17.074432 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" podUID="7f576ad1-48cd-469e-9c44-6a42322968e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 05:02:18.073782 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:18.073742 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 05:02:18.073996 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:18.073813 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" podUID="7f576ad1-48cd-469e-9c44-6a42322968e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 05:02:18.433117 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:18.433043 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="759be065-f27c-4886-8deb-1d56ced9139f" path="/var/lib/kubelet/pods/759be065-f27c-4886-8deb-1d56ced9139f/volumes" Nov 04 05:02:19.074369 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:19.074329 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 05:02:19.074533 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:19.074388 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" podUID="7f576ad1-48cd-469e-9c44-6a42322968e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 05:02:19.347433 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:19.347331 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:19.613744 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:19.613640 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:20.503171 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:20.503125 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:21.115067 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:21.115018 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:02:21.115588 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:21.115090 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" podUID="7f576ad1-48cd-469e-9c44-6a42322968e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:02:21.171863 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:21.171805 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:21.172390 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:21.172328 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:21.173133 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:21.173111 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:25.629894 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:25.623335 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:25.771557 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:25.771515 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:25.831307 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:02:25.831270 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 05:02:25.916757 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:25.916684 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:26.881036 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.881002 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm"] Nov 04 05:02:26.881440 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.881427 4374 cpu_manager.go:414] "containerMap: removing container" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerName="istio-proxy" Nov 04 05:02:26.881482 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.881440 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b8bd7f2-adbf-4f67-9f2a-3a3998f0135b" containerName="istio-proxy" Nov 04 05:02:26.881482 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.881450 4374 cpu_manager.go:414] "containerMap: removing container" podUID="418c3a14-fbcb-4566-9009-5d4799d59593" containerName="istio-proxy" Nov 04 05:02:26.881482 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.881455 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="418c3a14-fbcb-4566-9009-5d4799d59593" containerName="istio-proxy" Nov 04 05:02:26.881482 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.881469 4374 cpu_manager.go:414] "containerMap: removing container" podUID="f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" containerName="istio-proxy" Nov 04 05:02:26.881482 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.881474 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" containerName="istio-proxy" Nov 04 05:02:26.881631 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.881488 4374 cpu_manager.go:414] "containerMap: removing container" podUID="759be065-f27c-4886-8deb-1d56ced9139f" containerName="istio-proxy" Nov 04 05:02:26.881631 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.881493 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="759be065-f27c-4886-8deb-1d56ced9139f" containerName="istio-proxy" Nov 04 05:02:26.881631 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.881546 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="418c3a14-fbcb-4566-9009-5d4799d59593" containerName="istio-proxy" Nov 04 05:02:26.881631 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.881554 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="f269a2cb-cd4c-4f54-bbc4-d4a565f5d7da" containerName="istio-proxy" Nov 04 05:02:26.881631 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.881563 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="759be065-f27c-4886-8deb-1d56ced9139f" containerName="istio-proxy" Nov 04 05:02:26.886080 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.886060 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:26.975784 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.975749 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm"] Nov 04 05:02:26.979845 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.979816 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-credential-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:26.980002 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.979849 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c746255-9d07-4e19-ae6f-19a6763e4d76-istiod-ca-cert\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:26.980002 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.979958 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-data\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:26.980002 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.979987 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:26.980176 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.980022 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-certs\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:26.980176 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.980042 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5z6s\" (UniqueName: \"kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-kube-api-access-q5z6s\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:26.980176 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.980076 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-envoy\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:26.980176 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.980091 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-podinfo\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:26.980176 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:26.980124 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-token\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.081364 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.081326 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-data\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.081529 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.081374 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.081529 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.081420 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-certs\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.081529 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.081449 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5z6s\" (UniqueName: \"kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-kube-api-access-q5z6s\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.081529 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.081496 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-envoy\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.081529 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.081516 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-podinfo\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.081815 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.081563 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-token\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.081815 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.081647 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-credential-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.081815 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.081672 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c746255-9d07-4e19-ae6f-19a6763e4d76-istiod-ca-cert\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.082056 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.081837 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-certs\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.082056 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.081905 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.082056 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.082023 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-data\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.082213 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.082096 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-credential-socket\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.082420 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.082399 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c746255-9d07-4e19-ae6f-19a6763e4d76-istiod-ca-cert\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.083880 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.083845 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-envoy\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.084079 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.084062 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-podinfo\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.139553 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.139471 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5z6s\" (UniqueName: \"kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-kube-api-access-q5z6s\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.139848 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.139826 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-token\") pod \"e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.198699 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.198670 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:27.615942 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.615903 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm"] Nov 04 05:02:27.618446 ip-10-0-140-242 kubenswrapper[4374]: W1104 05:02:27.618415 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c746255_9d07_4e19_ae6f_19a6763e4d76.slice/crio-7cd6f36ed18c44e3684541ec1a1b58f64d73e564b2228d1eb244010ac0dd9221 WatchSource:0}: Error finding container 7cd6f36ed18c44e3684541ec1a1b58f64d73e564b2228d1eb244010ac0dd9221: Status 404 returned error can't find the container with id 7cd6f36ed18c44e3684541ec1a1b58f64d73e564b2228d1eb244010ac0dd9221 Nov 04 05:02:27.620880 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.620833 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:02:27.620963 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.620912 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:02:27.620963 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:27.620944 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:02:28.339044 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:28.339006 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" event={"ID":"2c746255-9d07-4e19-ae6f-19a6763e4d76","Type":"ContainerStarted","Data":"e661ff4b5cb19e01db80ed10e67ea5f7cd339807f5c52eba52bf648fe28b038f"} Nov 04 05:02:28.339433 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:28.339067 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" event={"ID":"2c746255-9d07-4e19-ae6f-19a6763e4d76","Type":"ContainerStarted","Data":"7cd6f36ed18c44e3684541ec1a1b58f64d73e564b2228d1eb244010ac0dd9221"} Nov 04 05:02:28.574439 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:28.574379 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" podStartSLOduration=2.574365299 podStartE2EDuration="2.574365299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:02:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:02:28.52635451 +0000 UTC m=+990.238045649" watchObservedRunningTime="2026-11-04 05:02:28.574365299 +0000 UTC m=+990.286056500" Nov 04 05:02:29.199171 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:29.199129 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:29.200761 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:29.200740 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Nov 04 05:02:29.200823 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:29.200786 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" podUID="2c746255-9d07-4e19-ae6f-19a6763e4d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Nov 04 05:02:29.931892 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:29.930741 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp"] Nov 04 05:02:29.931892 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:29.931035 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" podUID="c0cec3be-0cf8-4043-bb2e-3cf776d2ec07" containerName="maas-api" containerID="cri-o://c6972de3525e2743dd0de4cb892f5762641c17f3135a7d0e33c339b06e8a39bb" gracePeriod=30 Nov 04 05:02:30.030967 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:30.030796 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:30.199438 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:30.199353 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Nov 04 05:02:30.199438 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:30.199421 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" podUID="2c746255-9d07-4e19-ae6f-19a6763e4d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Nov 04 05:02:31.199191 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.199153 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Nov 04 05:02:31.199651 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.199221 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" podUID="2c746255-9d07-4e19-ae6f-19a6763e4d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Nov 04 05:02:31.367095 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.367012 4374 generic.go:356] "Generic (PLEG): container finished" podID="c0cec3be-0cf8-4043-bb2e-3cf776d2ec07" containerID="c6972de3525e2743dd0de4cb892f5762641c17f3135a7d0e33c339b06e8a39bb" exitCode=0 Nov 04 05:02:31.367236 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.367088 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" event={"ID":"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07","Type":"ContainerDied","Data":"c6972de3525e2743dd0de4cb892f5762641c17f3135a7d0e33c339b06e8a39bb"} Nov 04 05:02:31.545795 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.545769 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:31.688094 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.688007 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:31.775773 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.775746 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 05:02:31.867364 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.867333 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-kube-api-access-l4msv\" (UniqueName: \"kubernetes.io/projected/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-kube-api-access-l4msv\") pod \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " Nov 04 05:02:31.867546 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.867380 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-openshift-service-ca\") pod \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " Nov 04 05:02:31.867546 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.867418 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-metrics-certs\") pod \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " Nov 04 05:02:31.867546 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.867459 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-maas-api-tls\") pod \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\" (UID: \"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07\") " Nov 04 05:02:31.867816 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.867779 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-openshift-service-ca" pod "c0cec3be-0cf8-4043-bb2e-3cf776d2ec07" (UID: "c0cec3be-0cf8-4043-bb2e-3cf776d2ec07"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:02:31.869515 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.869490 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-kube-api-access-l4msv" pod "c0cec3be-0cf8-4043-bb2e-3cf776d2ec07" (UID: "c0cec3be-0cf8-4043-bb2e-3cf776d2ec07"). InnerVolumeSpecName "kube-api-access-l4msv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:31.869629 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.869604 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-maas-api-tls" pod "c0cec3be-0cf8-4043-bb2e-3cf776d2ec07" (UID: "c0cec3be-0cf8-4043-bb2e-3cf776d2ec07"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:02:31.870092 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.870077 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-metrics-certs" pod "c0cec3be-0cf8-4043-bb2e-3cf776d2ec07" (UID: "c0cec3be-0cf8-4043-bb2e-3cf776d2ec07"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:02:31.968481 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.968387 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4msv\" (UniqueName: \"kubernetes.io/projected/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-kube-api-access-l4msv\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:31.968481 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.968429 4374 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-openshift-service-ca\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:31.968481 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.968441 4374 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-metrics-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:31.968481 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:31.968460 4374 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07-maas-api-tls\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:32.199282 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:32.199245 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Nov 04 05:02:32.199681 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:32.199294 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" podUID="2c746255-9d07-4e19-ae6f-19a6763e4d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Nov 04 05:02:32.237123 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:32.237094 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:32.373156 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:32.373131 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" Nov 04 05:02:32.373334 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:32.373176 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp" event={"ID":"c0cec3be-0cf8-4043-bb2e-3cf776d2ec07","Type":"ContainerDied","Data":"08afbfb4e753d922c566b0638b9efca78edfe219cc5ee0f4ce1271513478c24f"} Nov 04 05:02:32.373334 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:32.373287 4374 scope.go:122] "RemoveContainer" containerID="c6972de3525e2743dd0de4cb892f5762641c17f3135a7d0e33c339b06e8a39bb" Nov 04 05:02:32.534072 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:32.533990 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp"] Nov 04 05:02:32.538089 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:32.538062 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c5bb6d-5b6888c9c9-l2njp"] Nov 04 05:02:33.007167 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:33.007134 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:33.199194 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:33.199164 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Nov 04 05:02:33.199348 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:33.199217 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" podUID="2c746255-9d07-4e19-ae6f-19a6763e4d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Nov 04 05:02:34.199915 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:34.199863 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Nov 04 05:02:34.200393 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:34.199941 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" podUID="2c746255-9d07-4e19-ae6f-19a6763e4d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Nov 04 05:02:34.412253 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:34.412214 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0cec3be-0cf8-4043-bb2e-3cf776d2ec07" path="/var/lib/kubelet/pods/c0cec3be-0cf8-4043-bb2e-3cf776d2ec07/volumes" Nov 04 05:02:36.243062 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:36.243018 4374 patch_prober.go:29] interesting pod/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:02:36.243618 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:36.243087 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" podUID="2c746255-9d07-4e19-ae6f-19a6763e4d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:02:36.365746 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:36.365718 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:36.366150 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:36.366127 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:36.366941 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:36.366924 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:37.627746 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:37.627706 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:38.306360 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:38.306329 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:38.462225 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:38.462192 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:39.060797 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:39.060760 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:41.480583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:41.480548 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:41.621511 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:41.621469 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:41.821192 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:41.821114 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx"] Nov 04 05:02:41.821357 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:41.821317 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" podUID="fff61915-de2e-41b2-91c0-5e608c0b1fcb" containerName="maas-api" containerID="cri-o://41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645" gracePeriod=30 Nov 04 05:02:41.933673 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:41.933644 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:42.375441 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.375392 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q"] Nov 04 05:02:42.377454 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.377424 4374 cpu_manager.go:414] "containerMap: removing container" podUID="c0cec3be-0cf8-4043-bb2e-3cf776d2ec07" containerName="maas-api" Nov 04 05:02:42.377454 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.377449 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0cec3be-0cf8-4043-bb2e-3cf776d2ec07" containerName="maas-api" Nov 04 05:02:42.377631 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.377568 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0cec3be-0cf8-4043-bb2e-3cf776d2ec07" containerName="maas-api" Nov 04 05:02:42.381082 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.381059 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.438680 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.438650 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q"] Nov 04 05:02:42.483339 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.483309 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-data\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.483810 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.483351 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-socket\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.483810 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.483436 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-certs\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.483810 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.483513 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f430f13-3f82-4097-ba3d-f9c047233147-istiod-ca-cert\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.483810 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.483574 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-credential-socket\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.483810 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.483634 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-istio-token\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.483810 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.483718 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhzzf\" (UniqueName: \"kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-kube-api-access-nhzzf\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.483810 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.483745 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f430f13-3f82-4097-ba3d-f9c047233147-istio-podinfo\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.484087 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.483816 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-envoy\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.584953 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.584910 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhzzf\" (UniqueName: \"kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-kube-api-access-nhzzf\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.584953 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.584952 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f430f13-3f82-4097-ba3d-f9c047233147-istio-podinfo\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.585215 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.584997 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-envoy\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.585276 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.585238 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-data\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.585354 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.585330 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-socket\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.585471 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.585444 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-certs\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.585565 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.585501 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f430f13-3f82-4097-ba3d-f9c047233147-istiod-ca-cert\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.585620 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.585584 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-data\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.585678 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.585612 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-credential-socket\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.585678 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.585652 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-istio-token\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.586189 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.586162 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f430f13-3f82-4097-ba3d-f9c047233147-istiod-ca-cert\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.586468 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.586448 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-certs\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.586550 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.586536 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-credential-socket\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.586590 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.586530 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-socket\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.587919 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.587902 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-envoy\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.588056 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.588034 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f430f13-3f82-4097-ba3d-f9c047233147-istio-podinfo\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.642519 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.642440 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhzzf\" (UniqueName: \"kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-kube-api-access-nhzzf\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.642848 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.642827 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-istio-token\") pod \"e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.694705 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.694669 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:42.750740 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.750709 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:42.895721 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.895633 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:42.968390 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:42.968349 4374 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" podUID="fff61915-de2e-41b2-91c0-5e608c0b1fcb" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.42:8443/health\": dial tcp 10.133.0.42:8443: connect: connection refused" Nov 04 05:02:43.064251 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.064226 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 05:02:43.117972 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.117942 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q"] Nov 04 05:02:43.119562 ip-10-0-140-242 kubenswrapper[4374]: W1104 05:02:43.119522 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f430f13_3f82_4097_ba3d_f9c047233147.slice/crio-bda4e4988414e6a08c669d7ad1a46c67261c49241b806c9f0bd97550b60f44c4 WatchSource:0}: Error finding container bda4e4988414e6a08c669d7ad1a46c67261c49241b806c9f0bd97550b60f44c4: Status 404 returned error can't find the container with id bda4e4988414e6a08c669d7ad1a46c67261c49241b806c9f0bd97550b60f44c4 Nov 04 05:02:43.121840 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.121809 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:02:43.121919 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.121888 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:02:43.121969 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.121920 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:02:43.192963 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.192933 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-maas-api-tls\") pod \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " Nov 04 05:02:43.193137 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.192995 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fff61915-de2e-41b2-91c0-5e608c0b1fcb-kube-api-access-mdvlx\" (UniqueName: \"kubernetes.io/projected/fff61915-de2e-41b2-91c0-5e608c0b1fcb-kube-api-access-mdvlx\") pod \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " Nov 04 05:02:43.193137 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.193040 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fff61915-de2e-41b2-91c0-5e608c0b1fcb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff61915-de2e-41b2-91c0-5e608c0b1fcb-openshift-service-ca\") pod \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " Nov 04 05:02:43.193137 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.193114 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-metrics-certs\") pod \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\" (UID: \"fff61915-de2e-41b2-91c0-5e608c0b1fcb\") " Nov 04 05:02:43.195452 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.195425 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fff61915-de2e-41b2-91c0-5e608c0b1fcb-openshift-service-ca" pod "fff61915-de2e-41b2-91c0-5e608c0b1fcb" (UID: "fff61915-de2e-41b2-91c0-5e608c0b1fcb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:02:43.195671 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.195651 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-maas-api-tls" pod "fff61915-de2e-41b2-91c0-5e608c0b1fcb" (UID: "fff61915-de2e-41b2-91c0-5e608c0b1fcb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:02:43.195854 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.195822 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-metrics-certs" pod "fff61915-de2e-41b2-91c0-5e608c0b1fcb" (UID: "fff61915-de2e-41b2-91c0-5e608c0b1fcb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:02:43.195998 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.195921 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fff61915-de2e-41b2-91c0-5e608c0b1fcb-kube-api-access-mdvlx" pod "fff61915-de2e-41b2-91c0-5e608c0b1fcb" (UID: "fff61915-de2e-41b2-91c0-5e608c0b1fcb"). InnerVolumeSpecName "kube-api-access-mdvlx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:43.297422 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.297384 4374 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-maas-api-tls\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:43.297422 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.297417 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdvlx\" (UniqueName: \"kubernetes.io/projected/fff61915-de2e-41b2-91c0-5e608c0b1fcb-kube-api-access-mdvlx\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:43.297422 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.297428 4374 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff61915-de2e-41b2-91c0-5e608c0b1fcb-openshift-service-ca\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:43.297660 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.297436 4374 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff61915-de2e-41b2-91c0-5e608c0b1fcb-metrics-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:43.435857 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.435764 4374 generic.go:356] "Generic (PLEG): container finished" podID="fff61915-de2e-41b2-91c0-5e608c0b1fcb" containerID="41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645" exitCode=0 Nov 04 05:02:43.435857 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.435814 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" Nov 04 05:02:43.436092 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.435850 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" event={"ID":"fff61915-de2e-41b2-91c0-5e608c0b1fcb","Type":"ContainerDied","Data":"41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645"} Nov 04 05:02:43.436092 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.435940 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx" event={"ID":"fff61915-de2e-41b2-91c0-5e608c0b1fcb","Type":"ContainerDied","Data":"1a7cebe80668b103663feeff8bb66c92a42d3c9200ceb8cab8f7e295f2cfba7c"} Nov 04 05:02:43.436092 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.435966 4374 scope.go:122] "RemoveContainer" containerID="41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645" Nov 04 05:02:43.438506 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.437789 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" event={"ID":"2f430f13-3f82-4097-ba3d-f9c047233147","Type":"ContainerStarted","Data":"2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb"} Nov 04 05:02:43.438506 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.437840 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" event={"ID":"2f430f13-3f82-4097-ba3d-f9c047233147","Type":"ContainerStarted","Data":"bda4e4988414e6a08c669d7ad1a46c67261c49241b806c9f0bd97550b60f44c4"} Nov 04 05:02:43.446319 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.446302 4374 scope.go:122] "RemoveContainer" containerID="41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645" Nov 04 05:02:43.446607 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:02:43.446586 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645\": container with ID starting with 41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645 not found: ID does not exist" containerID="41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645" Nov 04 05:02:43.446656 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.446616 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645"} err="failed to get container status \"41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645\": rpc error: code = NotFound desc = could not find container \"41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645\": container with ID starting with 41dee07b569c75e75e5d383e9f197d53db51e83f40fd20e56e03883873382645 not found: ID does not exist" Nov 04 05:02:43.589481 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.589447 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx"] Nov 04 05:02:43.593793 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.593760 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-204e38-5d98bb8b54-bwwbx"] Nov 04 05:02:43.695766 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.695688 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:43.705239 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.705190 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" podStartSLOduration=1.705176996 podStartE2EDuration="1.705176996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:02:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:02:43.70255828 +0000 UTC m=+1005.414249418" watchObservedRunningTime="2026-11-04 05:02:43.705176996 +0000 UTC m=+1005.416868196" Nov 04 05:02:43.815887 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:43.813333 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:44.470576 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:44.470544 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fff61915-de2e-41b2-91c0-5e608c0b1fcb" path="/var/lib/kubelet/pods/fff61915-de2e-41b2-91c0-5e608c0b1fcb/volumes" Nov 04 05:02:44.482110 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:44.482084 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:44.633907 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:44.633849 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 05:02:44.657277 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:44.657244 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:44.699217 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:44.698784 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 05:02:44.699217 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:44.698846 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" podUID="2f430f13-3f82-4097-ba3d-f9c047233147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 05:02:44.802012 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:44.801982 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:45.151903 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:45.151794 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp"] Nov 04 05:02:45.152047 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:45.152012 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" podUID="7f576ad1-48cd-469e-9c44-6a42322968e7" containerName="istio-proxy" containerID="cri-o://1ac2283a140ca82af1f52c4f08770864c6dc855f55985975fcfeddc2e77204c9" gracePeriod=30 Nov 04 05:02:45.160428 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:45.160393 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:45.181541 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:45.181517 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm"] Nov 04 05:02:45.181711 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:45.181693 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" podUID="2c746255-9d07-4e19-ae6f-19a6763e4d76" containerName="istio-proxy" containerID="cri-o://e661ff4b5cb19e01db80ed10e67ea5f7cd339807f5c52eba52bf648fe28b038f" gracePeriod=30 Nov 04 05:02:45.192068 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:45.192041 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99"] Nov 04 05:02:45.192271 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:45.192249 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerName="istio-proxy" containerID="cri-o://478f71e3be30523070819cdb23b35ccddde69b56045d9370b4d6d9a2f1fb3d4c" gracePeriod=30 Nov 04 05:02:45.302119 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:45.302072 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:45.695357 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:45.695318 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 05:02:45.695907 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:45.695374 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" podUID="2f430f13-3f82-4097-ba3d-f9c047233147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 05:02:45.832890 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:45.823348 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:45.966485 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:45.966383 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:46.695522 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:46.695487 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 05:02:46.695945 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:46.695540 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" podUID="2f430f13-3f82-4097-ba3d-f9c047233147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 05:02:47.695837 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:47.695805 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 05:02:47.696260 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:47.695863 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" podUID="2f430f13-3f82-4097-ba3d-f9c047233147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 05:02:48.695481 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:48.695446 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 05:02:48.695663 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:48.695502 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" podUID="2f430f13-3f82-4097-ba3d-f9c047233147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 05:02:49.093258 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:49.093229 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:49.695254 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:49.695223 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Nov 04 05:02:49.695470 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:49.695295 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" podUID="2f430f13-3f82-4097-ba3d-f9c047233147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Nov 04 05:02:50.312110 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.312083 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:50.514124 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.511594 4374 generic.go:356] "Generic (PLEG): container finished" podID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerID="478f71e3be30523070819cdb23b35ccddde69b56045d9370b4d6d9a2f1fb3d4c" exitCode=0 Nov 04 05:02:50.514124 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.511662 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" event={"ID":"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a","Type":"ContainerDied","Data":"478f71e3be30523070819cdb23b35ccddde69b56045d9370b4d6d9a2f1fb3d4c"} Nov 04 05:02:50.514124 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.511702 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" event={"ID":"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a","Type":"ContainerDied","Data":"c76c34c6439b6f3fbefd18460a18502b9b8e00b62a4ad93c39cb8dcf07959dbc"} Nov 04 05:02:50.514124 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.511719 4374 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c76c34c6439b6f3fbefd18460a18502b9b8e00b62a4ad93c39cb8dcf07959dbc" Nov 04 05:02:50.514124 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.511733 4374 scope.go:122] "RemoveContainer" containerID="5b58fbb0a4c71a48e636d82ea221531ded1e8d8d43d97c6fbbeedda95bb3d109" Nov 04 05:02:50.514942 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.514916 4374 generic.go:356] "Generic (PLEG): container finished" podID="7f576ad1-48cd-469e-9c44-6a42322968e7" containerID="1ac2283a140ca82af1f52c4f08770864c6dc855f55985975fcfeddc2e77204c9" exitCode=0 Nov 04 05:02:50.515106 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.515088 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" event={"ID":"7f576ad1-48cd-469e-9c44-6a42322968e7","Type":"ContainerDied","Data":"1ac2283a140ca82af1f52c4f08770864c6dc855f55985975fcfeddc2e77204c9"} Nov 04 05:02:50.516600 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.516578 4374 generic.go:356] "Generic (PLEG): container finished" podID="2c746255-9d07-4e19-ae6f-19a6763e4d76" containerID="e661ff4b5cb19e01db80ed10e67ea5f7cd339807f5c52eba52bf648fe28b038f" exitCode=0 Nov 04 05:02:50.516710 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.516614 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" event={"ID":"2c746255-9d07-4e19-ae6f-19a6763e4d76","Type":"ContainerDied","Data":"e661ff4b5cb19e01db80ed10e67ea5f7cd339807f5c52eba52bf648fe28b038f"} Nov 04 05:02:50.516710 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.516652 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" event={"ID":"2c746255-9d07-4e19-ae6f-19a6763e4d76","Type":"ContainerDied","Data":"7cd6f36ed18c44e3684541ec1a1b58f64d73e564b2228d1eb244010ac0dd9221"} Nov 04 05:02:50.516710 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.516675 4374 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cd6f36ed18c44e3684541ec1a1b58f64d73e564b2228d1eb244010ac0dd9221" Nov 04 05:02:50.517421 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.517343 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 05:02:50.521510 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.521491 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:50.551818 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.551757 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c746255-9d07-4e19-ae6f-19a6763e4d76-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c746255-9d07-4e19-ae6f-19a6763e4d76-istiod-ca-cert\") pod \"2c746255-9d07-4e19-ae6f-19a6763e4d76\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " Nov 04 05:02:50.551978 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.551821 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-podinfo\") pod \"2c746255-9d07-4e19-ae6f-19a6763e4d76\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " Nov 04 05:02:50.551978 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.551851 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-podinfo\") pod \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " Nov 04 05:02:50.551978 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.551925 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-token\" (UniqueName: \"kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-token\") pod \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " Nov 04 05:02:50.551978 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.551968 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istiod-ca-cert\") pod \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " Nov 04 05:02:50.552144 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.551996 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-socket\") pod \"2c746255-9d07-4e19-ae6f-19a6763e4d76\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " Nov 04 05:02:50.552144 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552045 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-credential-socket\") pod \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " Nov 04 05:02:50.552144 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552122 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-data\") pod \"2c746255-9d07-4e19-ae6f-19a6763e4d76\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " Nov 04 05:02:50.552270 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552161 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-data\") pod \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " Nov 04 05:02:50.552270 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552198 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-envoy\") pod \"2c746255-9d07-4e19-ae6f-19a6763e4d76\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " Nov 04 05:02:50.552270 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552248 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-credential-socket\") pod \"2c746255-9d07-4e19-ae6f-19a6763e4d76\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " Nov 04 05:02:50.552380 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552278 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-certs\") pod \"2c746255-9d07-4e19-ae6f-19a6763e4d76\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " Nov 04 05:02:50.552380 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552324 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-kube-api-access-q5z6s\" (UniqueName: \"kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-kube-api-access-q5z6s\") pod \"2c746255-9d07-4e19-ae6f-19a6763e4d76\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " Nov 04 05:02:50.552380 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552364 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-envoy\") pod \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " Nov 04 05:02:50.552489 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552402 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-kube-api-access-jf5gl\" (UniqueName: \"kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-kube-api-access-jf5gl\") pod \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " Nov 04 05:02:50.552489 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552414 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c746255-9d07-4e19-ae6f-19a6763e4d76-istiod-ca-cert" pod "2c746255-9d07-4e19-ae6f-19a6763e4d76" (UID: "2c746255-9d07-4e19-ae6f-19a6763e4d76"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:02:50.552489 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552457 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-certs\") pod \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " Nov 04 05:02:50.552589 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552517 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-socket\") pod \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\" (UID: \"12c89dac-0d97-4aea-b0fe-3e9a9502ae4a\") " Nov 04 05:02:50.552589 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552564 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-token\" (UniqueName: \"kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-token\") pod \"2c746255-9d07-4e19-ae6f-19a6763e4d76\" (UID: \"2c746255-9d07-4e19-ae6f-19a6763e4d76\") " Nov 04 05:02:50.552676 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.552659 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-data" pod "2c746255-9d07-4e19-ae6f-19a6763e4d76" (UID: "2c746255-9d07-4e19-ae6f-19a6763e4d76"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.553054 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.553029 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.553204 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.553188 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c746255-9d07-4e19-ae6f-19a6763e4d76-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.553407 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.553350 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istiod-ca-cert" pod "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" (UID: "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:02:50.554023 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.553529 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-credential-socket" pod "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" (UID: "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.554023 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.553663 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-credential-socket" pod "2c746255-9d07-4e19-ae6f-19a6763e4d76" (UID: "2c746255-9d07-4e19-ae6f-19a6763e4d76"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.554023 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.553704 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-certs" pod "2c746255-9d07-4e19-ae6f-19a6763e4d76" (UID: "2c746255-9d07-4e19-ae6f-19a6763e4d76"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.554023 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.553895 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-certs" pod "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" (UID: "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.554285 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.554092 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-socket" pod "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" (UID: "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.554285 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.554136 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-socket" pod "2c746255-9d07-4e19-ae6f-19a6763e4d76" (UID: "2c746255-9d07-4e19-ae6f-19a6763e4d76"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.554285 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.554174 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-data" pod "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" (UID: "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.557405 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.557375 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-podinfo" pod "2c746255-9d07-4e19-ae6f-19a6763e4d76" (UID: "2c746255-9d07-4e19-ae6f-19a6763e4d76"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:02:50.557501 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.557488 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-kube-api-access-jf5gl" pod "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" (UID: "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a"). InnerVolumeSpecName "kube-api-access-jf5gl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:50.557647 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.557611 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-envoy" pod "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" (UID: "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.557863 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.557831 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-kube-api-access-q5z6s" pod "2c746255-9d07-4e19-ae6f-19a6763e4d76" (UID: "2c746255-9d07-4e19-ae6f-19a6763e4d76"). InnerVolumeSpecName "kube-api-access-q5z6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:50.558807 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.558770 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-token" pod "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" (UID: "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:50.558807 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.558794 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-podinfo" pod "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" (UID: "12c89dac-0d97-4aea-b0fe-3e9a9502ae4a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:02:50.558985 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.558895 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-token" pod "2c746255-9d07-4e19-ae6f-19a6763e4d76" (UID: "2c746255-9d07-4e19-ae6f-19a6763e4d76"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:50.559685 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.559667 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:50.560102 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.560076 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-envoy" pod "2c746255-9d07-4e19-ae6f-19a6763e4d76" (UID: "2c746255-9d07-4e19-ae6f-19a6763e4d76"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.654297 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.654261 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-token\" (UniqueName: \"kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-token\") pod \"7f576ad1-48cd-469e-9c44-6a42322968e7\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " Nov 04 05:02:50.654470 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.654325 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-data\") pod \"7f576ad1-48cd-469e-9c44-6a42322968e7\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " Nov 04 05:02:50.654470 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.654378 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-credential-socket\") pod \"7f576ad1-48cd-469e-9c44-6a42322968e7\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " Nov 04 05:02:50.654608 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.654483 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-podinfo\") pod \"7f576ad1-48cd-469e-9c44-6a42322968e7\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " Nov 04 05:02:50.654608 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.654510 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-socket\") pod \"7f576ad1-48cd-469e-9c44-6a42322968e7\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " Nov 04 05:02:50.654608 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.654560 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-kube-api-access-qpgbl\" (UniqueName: \"kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-kube-api-access-qpgbl\") pod \"7f576ad1-48cd-469e-9c44-6a42322968e7\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " Nov 04 05:02:50.654781 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.654622 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-envoy\") pod \"7f576ad1-48cd-469e-9c44-6a42322968e7\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " Nov 04 05:02:50.654781 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.654658 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f576ad1-48cd-469e-9c44-6a42322968e7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f576ad1-48cd-469e-9c44-6a42322968e7-istiod-ca-cert\") pod \"7f576ad1-48cd-469e-9c44-6a42322968e7\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " Nov 04 05:02:50.654781 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.654658 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-data" pod "7f576ad1-48cd-469e-9c44-6a42322968e7" (UID: "7f576ad1-48cd-469e-9c44-6a42322968e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.654781 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.654702 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-certs\") pod \"7f576ad1-48cd-469e-9c44-6a42322968e7\" (UID: \"7f576ad1-48cd-469e-9c44-6a42322968e7\") " Nov 04 05:02:50.655057 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.654832 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-socket" pod "7f576ad1-48cd-469e-9c44-6a42322968e7" (UID: "7f576ad1-48cd-469e-9c44-6a42322968e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.655057 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.654952 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-credential-socket" pod "7f576ad1-48cd-469e-9c44-6a42322968e7" (UID: "7f576ad1-48cd-469e-9c44-6a42322968e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.655176 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655137 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655176 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655159 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655290 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655175 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5z6s\" (UniqueName: \"kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-kube-api-access-q5z6s\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655290 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655191 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655290 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655205 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655290 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655219 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf5gl\" (UniqueName: \"kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-kube-api-access-jf5gl\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655290 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655234 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655290 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655248 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655290 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655262 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655290 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655275 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655290 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655288 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655746 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655302 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655746 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655315 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655746 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655329 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655746 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655342 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655746 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655356 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655746 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655369 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655746 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655382 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.655746 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655395 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c746255-9d07-4e19-ae6f-19a6763e4d76-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.656179 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.655757 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f576ad1-48cd-469e-9c44-6a42322968e7-istiod-ca-cert" pod "7f576ad1-48cd-469e-9c44-6a42322968e7" (UID: "7f576ad1-48cd-469e-9c44-6a42322968e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:02:50.656326 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.656299 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-certs" pod "7f576ad1-48cd-469e-9c44-6a42322968e7" (UID: "7f576ad1-48cd-469e-9c44-6a42322968e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.657609 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.657580 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-kube-api-access-qpgbl" pod "7f576ad1-48cd-469e-9c44-6a42322968e7" (UID: "7f576ad1-48cd-469e-9c44-6a42322968e7"). InnerVolumeSpecName "kube-api-access-qpgbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:50.658363 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.658339 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-envoy" pod "7f576ad1-48cd-469e-9c44-6a42322968e7" (UID: "7f576ad1-48cd-469e-9c44-6a42322968e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:02:50.658977 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.658951 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-podinfo" pod "7f576ad1-48cd-469e-9c44-6a42322968e7" (UID: "7f576ad1-48cd-469e-9c44-6a42322968e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:02:50.659086 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.658998 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-token" pod "7f576ad1-48cd-469e-9c44-6a42322968e7" (UID: "7f576ad1-48cd-469e-9c44-6a42322968e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:02:50.756186 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.756135 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.756186 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.756172 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.756186 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.756181 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.756186 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.756190 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpgbl\" (UniqueName: \"kubernetes.io/projected/7f576ad1-48cd-469e-9c44-6a42322968e7-kube-api-access-qpgbl\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.756186 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.756199 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f576ad1-48cd-469e-9c44-6a42322968e7-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.756637 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.756207 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f576ad1-48cd-469e-9c44-6a42322968e7-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:02:50.869340 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.869257 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:50.869569 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.869546 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:50.870649 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:50.870618 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:02:51.525110 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:51.524714 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99" Nov 04 05:02:51.526581 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:51.526560 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" Nov 04 05:02:51.526709 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:51.526549 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp" event={"ID":"7f576ad1-48cd-469e-9c44-6a42322968e7","Type":"ContainerDied","Data":"c09c50bbf88dd64cafe3f30e583f8265dfdd63cabd7811881ff4053de824e0fb"} Nov 04 05:02:51.526785 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:51.526706 4374 scope.go:122] "RemoveContainer" containerID="1ac2283a140ca82af1f52c4f08770864c6dc855f55985975fcfeddc2e77204c9" Nov 04 05:02:51.526785 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:51.526762 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm" Nov 04 05:02:51.742917 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:51.742887 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm"] Nov 04 05:02:51.754002 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:51.753974 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-jxmdm"] Nov 04 05:02:51.856750 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:51.856675 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp"] Nov 04 05:02:51.862495 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:51.862466 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-6cbmp"] Nov 04 05:02:51.964652 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:51.964617 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99"] Nov 04 05:02:51.969128 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:51.969101 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c5bb6d-openshift-default-64f56dfd77-cst99"] Nov 04 05:02:52.411108 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:52.411071 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" path="/var/lib/kubelet/pods/12c89dac-0d97-4aea-b0fe-3e9a9502ae4a/volumes" Nov 04 05:02:52.411595 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:52.411580 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c746255-9d07-4e19-ae6f-19a6763e4d76" path="/var/lib/kubelet/pods/2c746255-9d07-4e19-ae6f-19a6763e4d76/volumes" Nov 04 05:02:52.412039 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:52.412026 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f576ad1-48cd-469e-9c44-6a42322968e7" path="/var/lib/kubelet/pods/7f576ad1-48cd-469e-9c44-6a42322968e7/volumes" Nov 04 05:02:53.995789 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:53.995752 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:55.831961 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:02:55.831921 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 05:02:56.478917 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.478887 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:56.895973 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.895898 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw"] Nov 04 05:02:56.896498 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896480 4374 cpu_manager.go:414] "containerMap: removing container" podUID="fff61915-de2e-41b2-91c0-5e608c0b1fcb" containerName="maas-api" Nov 04 05:02:56.896560 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896500 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fff61915-de2e-41b2-91c0-5e608c0b1fcb" containerName="maas-api" Nov 04 05:02:56.896560 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896516 4374 cpu_manager.go:414] "containerMap: removing container" podUID="2c746255-9d07-4e19-ae6f-19a6763e4d76" containerName="istio-proxy" Nov 04 05:02:56.896560 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896521 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c746255-9d07-4e19-ae6f-19a6763e4d76" containerName="istio-proxy" Nov 04 05:02:56.896560 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896539 4374 cpu_manager.go:414] "containerMap: removing container" podUID="7f576ad1-48cd-469e-9c44-6a42322968e7" containerName="istio-proxy" Nov 04 05:02:56.896560 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896545 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f576ad1-48cd-469e-9c44-6a42322968e7" containerName="istio-proxy" Nov 04 05:02:56.896560 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896558 4374 cpu_manager.go:414] "containerMap: removing container" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerName="istio-proxy" Nov 04 05:02:56.896909 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896563 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerName="istio-proxy" Nov 04 05:02:56.896909 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896570 4374 cpu_manager.go:414] "containerMap: removing container" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerName="istio-proxy" Nov 04 05:02:56.896909 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896575 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerName="istio-proxy" Nov 04 05:02:56.896909 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896655 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerName="istio-proxy" Nov 04 05:02:56.896909 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896667 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="12c89dac-0d97-4aea-b0fe-3e9a9502ae4a" containerName="istio-proxy" Nov 04 05:02:56.896909 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896680 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="fff61915-de2e-41b2-91c0-5e608c0b1fcb" containerName="maas-api" Nov 04 05:02:56.896909 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896691 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c746255-9d07-4e19-ae6f-19a6763e4d76" containerName="istio-proxy" Nov 04 05:02:56.896909 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.896700 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f576ad1-48cd-469e-9c44-6a42322968e7" containerName="istio-proxy" Nov 04 05:02:56.901311 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.901291 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:56.998804 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:56.998764 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw"] Nov 04 05:02:57.017173 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.017133 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-podinfo\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.017378 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.017219 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-envoy\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.017378 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.017247 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-data\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.017378 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.017286 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m8sx\" (UniqueName: \"kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-kube-api-access-5m8sx\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.017378 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.017340 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-token\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.017378 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.017367 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-credential-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.017761 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.017409 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istiod-ca-cert\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.017761 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.017440 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-certs\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.017761 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.017487 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.119115 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.119081 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-podinfo\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.119294 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.119136 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-envoy\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.119294 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.119157 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-data\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.119294 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.119277 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m8sx\" (UniqueName: \"kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-kube-api-access-5m8sx\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.119470 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.119338 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-token\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.119526 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.119503 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-credential-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.119583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.119524 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-data\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.119629 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.119592 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istiod-ca-cert\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.119677 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.119624 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-certs\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.119677 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.119671 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.119860 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.119829 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-credential-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.120022 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.119947 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-certs\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.120090 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.120030 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.120198 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.120178 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istiod-ca-cert\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.121423 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.121404 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-envoy\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.121662 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.121645 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-podinfo\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.176606 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.176539 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-token\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.176710 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.176695 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m8sx\" (UniqueName: \"kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-kube-api-access-5m8sx\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.214652 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.214624 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:57.399477 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.399447 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:57.541087 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.541052 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:57.647292 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.647266 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw"] Nov 04 05:02:57.649810 ip-10-0-140-242 kubenswrapper[4374]: W1104 05:02:57.649777 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea7cdff7_8f6a_420a_9b03_b3044df5fd28.slice/crio-c5bc4081bdb09f3bf86275263af0592ddd77ea04fbe5accefde955fbe27e87a2 WatchSource:0}: Error finding container c5bc4081bdb09f3bf86275263af0592ddd77ea04fbe5accefde955fbe27e87a2: Status 404 returned error can't find the container with id c5bc4081bdb09f3bf86275263af0592ddd77ea04fbe5accefde955fbe27e87a2 Nov 04 05:02:57.652090 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.652058 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:02:57.652155 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.652125 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:02:57.652197 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:57.652153 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:02:58.534021 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:58.533985 4374 scope.go:122] "RemoveContainer" containerID="d491c3f604368d4fabe98219170de4dcf5adda666cbb1fbe5443cf0555e6cda4" Nov 04 05:02:58.543563 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:58.543543 4374 scope.go:122] "RemoveContainer" containerID="478f71e3be30523070819cdb23b35ccddde69b56045d9370b4d6d9a2f1fb3d4c" Nov 04 05:02:58.563697 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:58.563667 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" event={"ID":"ea7cdff7-8f6a-420a-9b03-b3044df5fd28","Type":"ContainerStarted","Data":"16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1"} Nov 04 05:02:58.563799 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:58.563769 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" event={"ID":"ea7cdff7-8f6a-420a-9b03-b3044df5fd28","Type":"ContainerStarted","Data":"c5bc4081bdb09f3bf86275263af0592ddd77ea04fbe5accefde955fbe27e87a2"} Nov 04 05:02:58.727962 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:58.727897 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" podStartSLOduration=2.727862674 podStartE2EDuration="2.727862674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:02:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:02:58.727327828 +0000 UTC m=+1020.439018977" watchObservedRunningTime="2026-11-04 05:02:58.727862674 +0000 UTC m=+1020.439553814" Nov 04 05:02:59.002844 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:59.002809 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:02:59.226892 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:59.221707 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:02:59.226892 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:59.221855 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Nov 04 05:02:59.226892 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:02:59.221931 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Nov 04 05:03:00.227891 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:00.220253 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Nov 04 05:03:00.227891 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:00.220315 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Nov 04 05:03:00.424176 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:00.424137 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:00.883689 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:00.883657 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r"] Nov 04 05:03:00.884014 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:00.883860 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" podUID="7b02e827-6e2d-4eee-898a-15f8c815eb3d" containerName="istio-proxy" containerID="cri-o://2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c" gracePeriod=30 Nov 04 05:03:00.885641 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:00.885619 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q"] Nov 04 05:03:00.885742 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:00.885705 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:00.885815 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:00.885792 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" podUID="2f430f13-3f82-4097-ba3d-f9c047233147" containerName="istio-proxy" containerID="cri-o://2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb" gracePeriod=30 Nov 04 05:03:01.228899 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:01.228251 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Nov 04 05:03:01.228899 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:01.228315 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Nov 04 05:03:02.226893 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:02.221257 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Nov 04 05:03:02.226893 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:02.221319 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Nov 04 05:03:02.893820 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:02.893779 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:03:02.894216 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:02.893858 4374 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" podUID="7b02e827-6e2d-4eee-898a-15f8c815eb3d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:03:03.123326 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:03.123231 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:03.229017 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:03.228976 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Nov 04 05:03:03.229206 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:03.229053 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Nov 04 05:03:03.550111 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:03.550020 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:03.695582 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:03.695553 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:04.088140 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:04.088108 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:04.215399 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:04.215366 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Nov 04 05:03:04.215558 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:04.215419 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Nov 04 05:03:04.231860 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:04.231831 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:04.435236 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:04.435162 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:05.872580 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:05.872544 4374 patch_prober.go:29] interesting pod/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:03:05.872976 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:05.872608 4374 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" podUID="2f430f13-3f82-4097-ba3d-f9c047233147" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:03:06.171304 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.171280 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 05:03:06.188065 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.188043 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:03:06.223405 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.223369 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-envoy\") pod \"2f430f13-3f82-4097-ba3d-f9c047233147\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " Nov 04 05:03:06.223543 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.223434 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-socket\") pod \"2f430f13-3f82-4097-ba3d-f9c047233147\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " Nov 04 05:03:06.223543 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.223470 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-certs\") pod \"2f430f13-3f82-4097-ba3d-f9c047233147\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " Nov 04 05:03:06.223543 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.223501 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-istio-token\" (UniqueName: \"kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-istio-token\") pod \"2f430f13-3f82-4097-ba3d-f9c047233147\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " Nov 04 05:03:06.223543 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.223534 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-kube-api-access-9bvmx\" (UniqueName: \"kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-kube-api-access-9bvmx\") pod \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " Nov 04 05:03:06.223781 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.223578 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istiod-ca-cert\") pod \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " Nov 04 05:03:06.223781 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.223619 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-data\") pod \"2f430f13-3f82-4097-ba3d-f9c047233147\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " Nov 04 05:03:06.223924 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.223836 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-socket" pod "2f430f13-3f82-4097-ba3d-f9c047233147" (UID: "2f430f13-3f82-4097-ba3d-f9c047233147"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:06.223992 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.223917 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-certs\") pod \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " Nov 04 05:03:06.223992 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.223980 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-token\" (UniqueName: \"kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-token\") pod \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " Nov 04 05:03:06.224112 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224033 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-credential-socket\") pod \"2f430f13-3f82-4097-ba3d-f9c047233147\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " Nov 04 05:03:06.224112 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224080 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istiod-ca-cert" pod "7b02e827-6e2d-4eee-898a-15f8c815eb3d" (UID: "7b02e827-6e2d-4eee-898a-15f8c815eb3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:03:06.224112 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224087 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-podinfo\") pod \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " Nov 04 05:03:06.224283 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224152 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f430f13-3f82-4097-ba3d-f9c047233147-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f430f13-3f82-4097-ba3d-f9c047233147-istiod-ca-cert\") pod \"2f430f13-3f82-4097-ba3d-f9c047233147\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " Nov 04 05:03:06.224283 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224229 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2f430f13-3f82-4097-ba3d-f9c047233147-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f430f13-3f82-4097-ba3d-f9c047233147-istio-podinfo\") pod \"2f430f13-3f82-4097-ba3d-f9c047233147\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " Nov 04 05:03:06.224283 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224267 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-data\") pod \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " Nov 04 05:03:06.224450 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224375 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-certs" pod "2f430f13-3f82-4097-ba3d-f9c047233147" (UID: "2f430f13-3f82-4097-ba3d-f9c047233147"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:06.224450 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224415 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-socket\") pod \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " Nov 04 05:03:06.224557 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224472 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-kube-api-access-nhzzf\" (UniqueName: \"kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-kube-api-access-nhzzf\") pod \"2f430f13-3f82-4097-ba3d-f9c047233147\" (UID: \"2f430f13-3f82-4097-ba3d-f9c047233147\") " Nov 04 05:03:06.224557 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224539 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-credential-socket\") pod \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " Nov 04 05:03:06.224660 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224585 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-envoy\") pod \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\" (UID: \"7b02e827-6e2d-4eee-898a-15f8c815eb3d\") " Nov 04 05:03:06.225725 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224821 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-credential-socket" pod "2f430f13-3f82-4097-ba3d-f9c047233147" (UID: "2f430f13-3f82-4097-ba3d-f9c047233147"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:06.225725 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.224952 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-data" pod "2f430f13-3f82-4097-ba3d-f9c047233147" (UID: "2f430f13-3f82-4097-ba3d-f9c047233147"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:06.225725 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.225067 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.225725 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.225086 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.225725 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.225137 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.225725 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.225152 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.225725 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.225168 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.225725 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.225149 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-certs" pod "7b02e827-6e2d-4eee-898a-15f8c815eb3d" (UID: "7b02e827-6e2d-4eee-898a-15f8c815eb3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:06.225725 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.225341 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-data" pod "7b02e827-6e2d-4eee-898a-15f8c815eb3d" (UID: "7b02e827-6e2d-4eee-898a-15f8c815eb3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:06.225725 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.225435 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f430f13-3f82-4097-ba3d-f9c047233147-istiod-ca-cert" pod "2f430f13-3f82-4097-ba3d-f9c047233147" (UID: "2f430f13-3f82-4097-ba3d-f9c047233147"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:03:06.225725 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.225684 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-socket" pod "7b02e827-6e2d-4eee-898a-15f8c815eb3d" (UID: "7b02e827-6e2d-4eee-898a-15f8c815eb3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:06.226437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.225855 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-credential-socket" pod "7b02e827-6e2d-4eee-898a-15f8c815eb3d" (UID: "7b02e827-6e2d-4eee-898a-15f8c815eb3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:06.227915 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.227887 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-kube-api-access-9bvmx" pod "7b02e827-6e2d-4eee-898a-15f8c815eb3d" (UID: "7b02e827-6e2d-4eee-898a-15f8c815eb3d"). InnerVolumeSpecName "kube-api-access-9bvmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:03:06.228016 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.227989 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-podinfo" pod "7b02e827-6e2d-4eee-898a-15f8c815eb3d" (UID: "7b02e827-6e2d-4eee-898a-15f8c815eb3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:03:06.228267 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.228235 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-kube-api-access-nhzzf" pod "2f430f13-3f82-4097-ba3d-f9c047233147" (UID: "2f430f13-3f82-4097-ba3d-f9c047233147"). InnerVolumeSpecName "kube-api-access-nhzzf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:03:06.228377 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.228273 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-envoy" pod "2f430f13-3f82-4097-ba3d-f9c047233147" (UID: "2f430f13-3f82-4097-ba3d-f9c047233147"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:06.228377 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.228277 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-envoy" pod "7b02e827-6e2d-4eee-898a-15f8c815eb3d" (UID: "7b02e827-6e2d-4eee-898a-15f8c815eb3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:06.228785 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.228766 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-token" pod "7b02e827-6e2d-4eee-898a-15f8c815eb3d" (UID: "7b02e827-6e2d-4eee-898a-15f8c815eb3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:03:06.229063 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.229047 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f430f13-3f82-4097-ba3d-f9c047233147-istio-podinfo" pod "2f430f13-3f82-4097-ba3d-f9c047233147" (UID: "2f430f13-3f82-4097-ba3d-f9c047233147"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:03:06.229749 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.229732 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-istio-token" pod "2f430f13-3f82-4097-ba3d-f9c047233147" (UID: "2f430f13-3f82-4097-ba3d-f9c047233147"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:03:06.257045 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.256996 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:03:06.257193 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.257072 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:03:06.326189 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326156 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.326189 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326186 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.326189 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326196 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f430f13-3f82-4097-ba3d-f9c047233147-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.326477 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326205 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f430f13-3f82-4097-ba3d-f9c047233147-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.326477 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326215 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.326477 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326224 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.326477 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326235 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhzzf\" (UniqueName: \"kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-kube-api-access-nhzzf\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.326477 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326250 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.326477 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326262 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.326477 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326270 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f430f13-3f82-4097-ba3d-f9c047233147-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.326477 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326278 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f430f13-3f82-4097-ba3d-f9c047233147-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.326477 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326286 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bvmx\" (UniqueName: \"kubernetes.io/projected/7b02e827-6e2d-4eee-898a-15f8c815eb3d-kube-api-access-9bvmx\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.326477 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.326294 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b02e827-6e2d-4eee-898a-15f8c815eb3d-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:06.347040 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.347008 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:03:06.347748 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.347583 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:03:06.348799 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.348773 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:03:06.608934 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.608895 4374 generic.go:356] "Generic (PLEG): container finished" podID="2f430f13-3f82-4097-ba3d-f9c047233147" containerID="2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb" exitCode=0 Nov 04 05:03:06.608934 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.608932 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" Nov 04 05:03:06.609177 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.608959 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" event={"ID":"2f430f13-3f82-4097-ba3d-f9c047233147","Type":"ContainerDied","Data":"2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb"} Nov 04 05:03:06.609177 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.609029 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q" event={"ID":"2f430f13-3f82-4097-ba3d-f9c047233147","Type":"ContainerDied","Data":"bda4e4988414e6a08c669d7ad1a46c67261c49241b806c9f0bd97550b60f44c4"} Nov 04 05:03:06.609177 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.609060 4374 scope.go:122] "RemoveContainer" containerID="2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb" Nov 04 05:03:06.610523 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.610499 4374 generic.go:356] "Generic (PLEG): container finished" podID="7b02e827-6e2d-4eee-898a-15f8c815eb3d" containerID="2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c" exitCode=0 Nov 04 05:03:06.610654 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.610585 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" Nov 04 05:03:06.610654 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.610623 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" event={"ID":"7b02e827-6e2d-4eee-898a-15f8c815eb3d","Type":"ContainerDied","Data":"2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c"} Nov 04 05:03:06.610770 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.610666 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r" event={"ID":"7b02e827-6e2d-4eee-898a-15f8c815eb3d","Type":"ContainerDied","Data":"144767d439d001cfbd20b18d825f97ae5c88bd5360324039290dfb3469f1598c"} Nov 04 05:03:06.618761 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.618731 4374 scope.go:122] "RemoveContainer" containerID="2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb" Nov 04 05:03:06.619074 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:03:06.619045 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb\": container with ID starting with 2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb not found: ID does not exist" containerID="2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb" Nov 04 05:03:06.619162 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.619085 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb"} err="failed to get container status \"2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb\": rpc error: code = NotFound desc = could not find container \"2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb\": container with ID starting with 2c1f5462a57e0d3c883d215f1dcf23c54c333cdc4459f3be3d9fcbd65a1c36eb not found: ID does not exist" Nov 04 05:03:06.619162 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.619101 4374 scope.go:122] "RemoveContainer" containerID="2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c" Nov 04 05:03:06.629092 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.629072 4374 scope.go:122] "RemoveContainer" containerID="2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c" Nov 04 05:03:06.629435 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:03:06.629396 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c\": container with ID starting with 2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c not found: ID does not exist" containerID="2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c" Nov 04 05:03:06.629530 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.629434 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c"} err="failed to get container status \"2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c\": rpc error: code = NotFound desc = could not find container \"2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c\": container with ID starting with 2e138e961eeb94d50f98ebbb4a3014993455b085992f7b40a394ecc2d332167c not found: ID does not exist" Nov 04 05:03:06.763380 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.763346 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r"] Nov 04 05:03:06.767743 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.767716 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-mnq6r"] Nov 04 05:03:06.869786 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.869755 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q"] Nov 04 05:03:06.873533 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:06.873498 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-204e38-openshift-default-6d488f5d7f-lbv2q"] Nov 04 05:03:08.411146 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:08.411069 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f430f13-3f82-4097-ba3d-f9c047233147" path="/var/lib/kubelet/pods/2f430f13-3f82-4097-ba3d-f9c047233147/volumes" Nov 04 05:03:08.411530 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:08.411516 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b02e827-6e2d-4eee-898a-15f8c815eb3d" path="/var/lib/kubelet/pods/7b02e827-6e2d-4eee-898a-15f8c815eb3d/volumes" Nov 04 05:03:08.640154 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:08.640092 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:08.783669 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:08.783637 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:09.296634 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:09.296587 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:12.045590 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.045551 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m"] Nov 04 05:03:12.046032 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.046018 4374 cpu_manager.go:414] "containerMap: removing container" podUID="7b02e827-6e2d-4eee-898a-15f8c815eb3d" containerName="istio-proxy" Nov 04 05:03:12.046032 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.046031 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b02e827-6e2d-4eee-898a-15f8c815eb3d" containerName="istio-proxy" Nov 04 05:03:12.046103 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.046047 4374 cpu_manager.go:414] "containerMap: removing container" podUID="2f430f13-3f82-4097-ba3d-f9c047233147" containerName="istio-proxy" Nov 04 05:03:12.046103 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.046052 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f430f13-3f82-4097-ba3d-f9c047233147" containerName="istio-proxy" Nov 04 05:03:12.046173 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.046122 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b02e827-6e2d-4eee-898a-15f8c815eb3d" containerName="istio-proxy" Nov 04 05:03:12.046173 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.046131 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f430f13-3f82-4097-ba3d-f9c047233147" containerName="istio-proxy" Nov 04 05:03:12.050837 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.050814 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.188007 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.187973 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-podinfo\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.188007 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.188010 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxdk2\" (UniqueName: \"kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-kube-api-access-nxdk2\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.188238 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.188041 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-credential-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.188238 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.188146 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-token\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.188238 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.188178 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-certs\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.188238 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.188194 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-data\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.188238 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.188223 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-envoy\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.188405 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.188289 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istiod-ca-cert\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.188405 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.188329 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.247169 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.247137 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m"] Nov 04 05:03:12.288895 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.288848 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-envoy\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.288895 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.288896 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istiod-ca-cert\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.289135 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.288942 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.289135 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.288990 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-podinfo\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.289135 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.289012 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxdk2\" (UniqueName: \"kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-kube-api-access-nxdk2\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.289135 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.289043 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-credential-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.289135 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.289072 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-token\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.289135 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.289116 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-certs\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.289445 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.289141 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-data\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.289504 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.289447 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-credential-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.289560 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.289495 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-socket\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.289560 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.289525 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-data\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.289667 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.289610 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-certs\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.289771 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.289748 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istiod-ca-cert\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.291233 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.291207 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-envoy\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.291601 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.291582 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-podinfo\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.346540 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.346481 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-token\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.346663 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.346576 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxdk2\" (UniqueName: \"kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-kube-api-access-nxdk2\") pod \"e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.365230 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.365198 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:12.782090 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.782066 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m"] Nov 04 05:03:12.784579 ip-10-0-140-242 kubenswrapper[4374]: W1104 05:03:12.784549 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b1e71b_fa07_4881_824a_8b02f19bc2eb.slice/crio-16c1c0eab665a3dbebbcf4bba8a2364ef7516457d4245496f163a6501667ace0 WatchSource:0}: Error finding container 16c1c0eab665a3dbebbcf4bba8a2364ef7516457d4245496f163a6501667ace0: Status 404 returned error can't find the container with id 16c1c0eab665a3dbebbcf4bba8a2364ef7516457d4245496f163a6501667ace0 Nov 04 05:03:12.786674 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.786634 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:03:12.786760 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.786700 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:03:12.786760 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:12.786732 4374 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Nov 04 05:03:13.650179 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:13.650144 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" event={"ID":"81b1e71b-fa07-4881-824a-8b02f19bc2eb","Type":"ContainerStarted","Data":"d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895"} Nov 04 05:03:13.650592 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:13.650204 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" event={"ID":"81b1e71b-fa07-4881-824a-8b02f19bc2eb","Type":"ContainerStarted","Data":"16c1c0eab665a3dbebbcf4bba8a2364ef7516457d4245496f163a6501667ace0"} Nov 04 05:03:13.857209 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:13.857152 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" podStartSLOduration=2.857138145 podStartE2EDuration="2.857138145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:03:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:03:13.810020503 +0000 UTC m=+1035.521711670" watchObservedRunningTime="2026-11-04 05:03:13.857138145 +0000 UTC m=+1035.568829287" Nov 04 05:03:14.037092 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.037054 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l"] Nov 04 05:03:14.037308 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.037280 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" podUID="ad8b2336-652f-4e73-ab74-4a6c69aa5bc5" containerName="maas-api" containerID="cri-o://f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f" gracePeriod=30 Nov 04 05:03:14.145972 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.145941 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:14.337261 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.337236 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 05:03:14.366213 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.366187 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:14.367264 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.367249 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Nov 04 05:03:14.367316 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.367282 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Nov 04 05:03:14.440916 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.439215 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-maas-api-tls\") pod \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " Nov 04 05:03:14.440916 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.439272 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-kube-api-access-svhvc\" (UniqueName: \"kubernetes.io/projected/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-kube-api-access-svhvc\") pod \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " Nov 04 05:03:14.440916 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.439345 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-metrics-certs\") pod \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " Nov 04 05:03:14.440916 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.439405 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-openshift-service-ca\") pod \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\" (UID: \"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5\") " Nov 04 05:03:14.440916 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.440291 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-openshift-service-ca" pod "ad8b2336-652f-4e73-ab74-4a6c69aa5bc5" (UID: "ad8b2336-652f-4e73-ab74-4a6c69aa5bc5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:03:14.447909 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.447302 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-maas-api-tls" pod "ad8b2336-652f-4e73-ab74-4a6c69aa5bc5" (UID: "ad8b2336-652f-4e73-ab74-4a6c69aa5bc5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:03:14.449115 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.449092 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-metrics-certs" pod "ad8b2336-652f-4e73-ab74-4a6c69aa5bc5" (UID: "ad8b2336-652f-4e73-ab74-4a6c69aa5bc5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:03:14.464616 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.464573 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-kube-api-access-svhvc" pod "ad8b2336-652f-4e73-ab74-4a6c69aa5bc5" (UID: "ad8b2336-652f-4e73-ab74-4a6c69aa5bc5"). InnerVolumeSpecName "kube-api-access-svhvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:03:14.541639 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.540776 4374 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-maas-api-tls\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:14.541639 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.540804 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svhvc\" (UniqueName: \"kubernetes.io/projected/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-kube-api-access-svhvc\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:14.541639 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.540816 4374 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-metrics-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:14.541639 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.540826 4374 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5-openshift-service-ca\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:14.656600 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.656521 4374 generic.go:356] "Generic (PLEG): container finished" podID="ad8b2336-652f-4e73-ab74-4a6c69aa5bc5" containerID="f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f" exitCode=0 Nov 04 05:03:14.657140 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.657124 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" Nov 04 05:03:14.657977 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.657950 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" event={"ID":"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5","Type":"ContainerDied","Data":"f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f"} Nov 04 05:03:14.658037 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.658013 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l" event={"ID":"ad8b2336-652f-4e73-ab74-4a6c69aa5bc5","Type":"ContainerDied","Data":"63c0c5ed7bbb06cda857a22be02bb9d8ed22a7592417e751b81e702190d4b151"} Nov 04 05:03:14.658079 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.658043 4374 scope.go:122] "RemoveContainer" containerID="f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f" Nov 04 05:03:14.667012 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.666992 4374 scope.go:122] "RemoveContainer" containerID="f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f" Nov 04 05:03:14.667302 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:03:14.667284 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f\": container with ID starting with f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f not found: ID does not exist" containerID="f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f" Nov 04 05:03:14.667349 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.667312 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f"} err="failed to get container status \"f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f\": rpc error: code = NotFound desc = could not find container \"f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f\": container with ID starting with f4b792ac199dc9c02a6c0875fba16052102ad077f380c6f6a0df01d39a57cd0f not found: ID does not exist" Nov 04 05:03:14.829909 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.828298 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l"] Nov 04 05:03:14.832845 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:14.832809 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a48f7e-58f9475b6d-sfr8l"] Nov 04 05:03:15.279859 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:15.279830 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:15.365784 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:15.365749 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Nov 04 05:03:15.365974 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:15.365825 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Nov 04 05:03:16.365889 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:16.365836 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Nov 04 05:03:16.366301 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:16.365907 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Nov 04 05:03:16.434911 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:16.433421 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad8b2336-652f-4e73-ab74-4a6c69aa5bc5" path="/var/lib/kubelet/pods/ad8b2336-652f-4e73-ab74-4a6c69aa5bc5/volumes" Nov 04 05:03:17.366261 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:17.366229 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Nov 04 05:03:17.366669 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:17.366292 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Nov 04 05:03:18.365986 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:18.365953 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Nov 04 05:03:18.366184 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:18.366000 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Nov 04 05:03:19.366414 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:19.366384 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Nov 04 05:03:19.366806 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:19.366457 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Nov 04 05:03:20.831812 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:20.831763 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:20.973746 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:20.973715 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:21.367324 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:21.367287 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:03:21.367500 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:21.367342 4374 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:03:21.412732 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:21.412700 4374 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:21.413127 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:21.413104 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:21.414018 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:21.413990 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:25.832697 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:03:25.832657 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 05:03:27.311722 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:27.311687 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:27.495233 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:27.495204 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:27.571575 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:27.571491 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-r5g6w"] Nov 04 05:03:27.646027 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:27.645999 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:27.874400 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:27.874313 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:31.205712 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:31.205676 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:31.757946 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:31.757916 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m"] Nov 04 05:03:31.758148 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:31.758126 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerName="istio-proxy" containerID="cri-o://d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895" gracePeriod=30 Nov 04 05:03:31.760045 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:31.760017 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:31.767472 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:31.767447 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2"] Nov 04 05:03:31.767636 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:31.767618 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" podUID="51da6320-a68e-43f4-8a66-feff5075e34b" containerName="istio-proxy" containerID="cri-o://5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746" gracePeriod=30 Nov 04 05:03:31.779165 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:31.779143 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw"] Nov 04 05:03:31.779315 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:31.779298 4374 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerName="istio-proxy" containerID="cri-o://16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1" gracePeriod=30 Nov 04 05:03:33.115402 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:33.115364 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:03:33.115961 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:33.115428 4374 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" podUID="51da6320-a68e-43f4-8a66-feff5075e34b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:03:33.758923 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:33.758834 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:33.913237 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:33.913208 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:36.349823 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:36.349792 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:03:36.350216 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:36.349849 4374 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:03:36.415358 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:36.415323 4374 patch_prober.go:29] interesting pod/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Nov 04 05:03:36.415506 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:36.415385 4374 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:03:37.047094 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.047065 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:37.062433 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.062410 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:03:37.078312 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.078294 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:03:37.153826 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.153803 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51da6320-a68e-43f4-8a66-feff5075e34b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51da6320-a68e-43f4-8a66-feff5075e34b-istiod-ca-cert\") pod \"51da6320-a68e-43f4-8a66-feff5075e34b\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " Nov 04 05:03:37.153961 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.153838 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-token\" (UniqueName: \"kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-token\") pod \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " Nov 04 05:03:37.153961 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.153888 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-kube-api-access-ww6cf\" (UniqueName: \"kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-kube-api-access-ww6cf\") pod \"51da6320-a68e-43f4-8a66-feff5075e34b\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " Nov 04 05:03:37.153961 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.153921 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-podinfo\") pod \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " Nov 04 05:03:37.154129 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.153960 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-podinfo\") pod \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " Nov 04 05:03:37.154129 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.153996 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-credential-socket\") pod \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " Nov 04 05:03:37.154129 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154025 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-envoy\") pod \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " Nov 04 05:03:37.154129 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154057 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-envoy\") pod \"51da6320-a68e-43f4-8a66-feff5075e34b\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " Nov 04 05:03:37.154129 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154107 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-credential-socket\") pod \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " Nov 04 05:03:37.154407 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154142 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-credential-socket\") pod \"51da6320-a68e-43f4-8a66-feff5075e34b\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " Nov 04 05:03:37.154407 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154179 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/51da6320-a68e-43f4-8a66-feff5075e34b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51da6320-a68e-43f4-8a66-feff5075e34b-istio-podinfo\") pod \"51da6320-a68e-43f4-8a66-feff5075e34b\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " Nov 04 05:03:37.154407 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154210 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-certs\") pod \"51da6320-a68e-43f4-8a66-feff5075e34b\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " Nov 04 05:03:37.154407 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154238 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-socket\") pod \"51da6320-a68e-43f4-8a66-feff5075e34b\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " Nov 04 05:03:37.154407 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154291 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-token\" (UniqueName: \"kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-token\") pod \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " Nov 04 05:03:37.154407 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154320 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-data\") pod \"51da6320-a68e-43f4-8a66-feff5075e34b\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " Nov 04 05:03:37.154407 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154349 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-certs\") pod \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " Nov 04 05:03:37.154407 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154406 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-istio-token\" (UniqueName: \"kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-istio-token\") pod \"51da6320-a68e-43f4-8a66-feff5075e34b\" (UID: \"51da6320-a68e-43f4-8a66-feff5075e34b\") " Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154426 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51da6320-a68e-43f4-8a66-feff5075e34b-istiod-ca-cert" pod "51da6320-a68e-43f4-8a66-feff5075e34b" (UID: "51da6320-a68e-43f4-8a66-feff5075e34b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154446 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-data\") pod \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154445 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-credential-socket" pod "81b1e71b-fa07-4881-824a-8b02f19bc2eb" (UID: "81b1e71b-fa07-4881-824a-8b02f19bc2eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154478 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-kube-api-access-5m8sx\" (UniqueName: \"kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-kube-api-access-5m8sx\") pod \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154513 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-kube-api-access-nxdk2\" (UniqueName: \"kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-kube-api-access-nxdk2\") pod \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154572 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-socket\") pod \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154598 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-envoy\") pod \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154626 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istiod-ca-cert\") pod \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154653 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-socket\") pod \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\" (UID: \"81b1e71b-fa07-4881-824a-8b02f19bc2eb\") " Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154658 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-socket" pod "51da6320-a68e-43f4-8a66-feff5075e34b" (UID: "51da6320-a68e-43f4-8a66-feff5075e34b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154673 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-credential-socket" pod "ea7cdff7-8f6a-420a-9b03-b3044df5fd28" (UID: "ea7cdff7-8f6a-420a-9b03-b3044df5fd28"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154683 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-data\") pod \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154726 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istiod-ca-cert\") pod \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " Nov 04 05:03:37.154819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154767 4374 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-certs\") pod \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\" (UID: \"ea7cdff7-8f6a-420a-9b03-b3044df5fd28\") " Nov 04 05:03:37.155583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.154898 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-credential-socket" pod "51da6320-a68e-43f4-8a66-feff5075e34b" (UID: "51da6320-a68e-43f4-8a66-feff5075e34b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.155583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.155105 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-certs" pod "ea7cdff7-8f6a-420a-9b03-b3044df5fd28" (UID: "ea7cdff7-8f6a-420a-9b03-b3044df5fd28"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.155583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.155231 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.155583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.155248 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.155583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.155264 4374 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-credential-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.155583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.155277 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.155583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.155319 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.155583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.155333 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51da6320-a68e-43f4-8a66-feff5075e34b-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.155583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.155346 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-certs" pod "81b1e71b-fa07-4881-824a-8b02f19bc2eb" (UID: "81b1e71b-fa07-4881-824a-8b02f19bc2eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.155583 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.155484 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-socket" pod "ea7cdff7-8f6a-420a-9b03-b3044df5fd28" (UID: "ea7cdff7-8f6a-420a-9b03-b3044df5fd28"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.156141 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.155975 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-certs" pod "51da6320-a68e-43f4-8a66-feff5075e34b" (UID: "51da6320-a68e-43f4-8a66-feff5075e34b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.156141 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.156023 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-socket" pod "81b1e71b-fa07-4881-824a-8b02f19bc2eb" (UID: "81b1e71b-fa07-4881-824a-8b02f19bc2eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.156478 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.156322 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-data" pod "81b1e71b-fa07-4881-824a-8b02f19bc2eb" (UID: "81b1e71b-fa07-4881-824a-8b02f19bc2eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.156478 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.156331 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istiod-ca-cert" pod "81b1e71b-fa07-4881-824a-8b02f19bc2eb" (UID: "81b1e71b-fa07-4881-824a-8b02f19bc2eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:03:37.156478 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.156338 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-data" pod "51da6320-a68e-43f4-8a66-feff5075e34b" (UID: "51da6320-a68e-43f4-8a66-feff5075e34b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.156742 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.156609 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-data" pod "ea7cdff7-8f6a-420a-9b03-b3044df5fd28" (UID: "ea7cdff7-8f6a-420a-9b03-b3044df5fd28"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.156890 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.156775 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istiod-ca-cert" pod "ea7cdff7-8f6a-420a-9b03-b3044df5fd28" (UID: "ea7cdff7-8f6a-420a-9b03-b3044df5fd28"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:03:37.157827 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.157789 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-podinfo" pod "ea7cdff7-8f6a-420a-9b03-b3044df5fd28" (UID: "ea7cdff7-8f6a-420a-9b03-b3044df5fd28"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:03:37.158273 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.158032 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51da6320-a68e-43f4-8a66-feff5075e34b-istio-podinfo" pod "51da6320-a68e-43f4-8a66-feff5075e34b" (UID: "51da6320-a68e-43f4-8a66-feff5075e34b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:03:37.158273 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.158110 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-token" pod "81b1e71b-fa07-4881-824a-8b02f19bc2eb" (UID: "81b1e71b-fa07-4881-824a-8b02f19bc2eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:03:37.158273 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.158161 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-podinfo" pod "81b1e71b-fa07-4881-824a-8b02f19bc2eb" (UID: "81b1e71b-fa07-4881-824a-8b02f19bc2eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:03:37.158273 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.158226 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-envoy" pod "51da6320-a68e-43f4-8a66-feff5075e34b" (UID: "51da6320-a68e-43f4-8a66-feff5075e34b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.158273 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.158263 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-kube-api-access-ww6cf" pod "51da6320-a68e-43f4-8a66-feff5075e34b" (UID: "51da6320-a68e-43f4-8a66-feff5075e34b"). InnerVolumeSpecName "kube-api-access-ww6cf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:03:37.158587 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.158492 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-envoy" pod "ea7cdff7-8f6a-420a-9b03-b3044df5fd28" (UID: "ea7cdff7-8f6a-420a-9b03-b3044df5fd28"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.158971 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.158949 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-token" pod "ea7cdff7-8f6a-420a-9b03-b3044df5fd28" (UID: "ea7cdff7-8f6a-420a-9b03-b3044df5fd28"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:03:37.159059 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.159038 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-envoy" pod "81b1e71b-fa07-4881-824a-8b02f19bc2eb" (UID: "81b1e71b-fa07-4881-824a-8b02f19bc2eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:03:37.159102 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.159053 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-istio-token" pod "51da6320-a68e-43f4-8a66-feff5075e34b" (UID: "51da6320-a68e-43f4-8a66-feff5075e34b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:03:37.159102 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.159053 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-kube-api-access-5m8sx" pod "ea7cdff7-8f6a-420a-9b03-b3044df5fd28" (UID: "ea7cdff7-8f6a-420a-9b03-b3044df5fd28"). InnerVolumeSpecName "kube-api-access-5m8sx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:03:37.159176 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.159121 4374 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-kube-api-access-nxdk2" pod "81b1e71b-fa07-4881-824a-8b02f19bc2eb" (UID: "81b1e71b-fa07-4881-824a-8b02f19bc2eb"). InnerVolumeSpecName "kube-api-access-nxdk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:03:37.256167 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256138 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256167 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256164 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51da6320-a68e-43f4-8a66-feff5075e34b-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256167 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256175 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256184 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256192 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51da6320-a68e-43f4-8a66-feff5075e34b-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256201 4374 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-certs\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256209 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256218 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256226 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m8sx\" (UniqueName: \"kubernetes.io/projected/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-kube-api-access-5m8sx\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256241 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxdk2\" (UniqueName: \"kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-kube-api-access-nxdk2\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256255 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256270 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256283 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256296 4374 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-workload-socket\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256308 4374 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-data\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256322 4374 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istiod-ca-cert\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256335 4374 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-token\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256343 4374 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww6cf\" (UniqueName: \"kubernetes.io/projected/51da6320-a68e-43f4-8a66-feff5075e34b-kube-api-access-ww6cf\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256351 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256363 4374 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7cdff7-8f6a-420a-9b03-b3044df5fd28-istio-podinfo\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.256437 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.256374 4374 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b1e71b-fa07-4881-824a-8b02f19bc2eb-istio-envoy\") on node \"ip-10-0-140-242.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:03:37.773260 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.773226 4374 generic.go:356] "Generic (PLEG): container finished" podID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerID="16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1" exitCode=0 Nov 04 05:03:37.773738 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.773274 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" Nov 04 05:03:37.773738 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.773309 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" event={"ID":"ea7cdff7-8f6a-420a-9b03-b3044df5fd28","Type":"ContainerDied","Data":"16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1"} Nov 04 05:03:37.773738 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.773373 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw" event={"ID":"ea7cdff7-8f6a-420a-9b03-b3044df5fd28","Type":"ContainerDied","Data":"c5bc4081bdb09f3bf86275263af0592ddd77ea04fbe5accefde955fbe27e87a2"} Nov 04 05:03:37.773738 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.773410 4374 scope.go:122] "RemoveContainer" containerID="16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1" Nov 04 05:03:37.774790 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.774763 4374 generic.go:356] "Generic (PLEG): container finished" podID="51da6320-a68e-43f4-8a66-feff5075e34b" containerID="5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746" exitCode=0 Nov 04 05:03:37.774933 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.774822 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" Nov 04 05:03:37.774933 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.774854 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" event={"ID":"51da6320-a68e-43f4-8a66-feff5075e34b","Type":"ContainerDied","Data":"5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746"} Nov 04 05:03:37.775042 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.774934 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2" event={"ID":"51da6320-a68e-43f4-8a66-feff5075e34b","Type":"ContainerDied","Data":"6c03c2455a0131e102c78e80311ac5267b4700a08893edd1616441849527807b"} Nov 04 05:03:37.776430 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.776401 4374 generic.go:356] "Generic (PLEG): container finished" podID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerID="d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895" exitCode=0 Nov 04 05:03:37.776521 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.776456 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" event={"ID":"81b1e71b-fa07-4881-824a-8b02f19bc2eb","Type":"ContainerDied","Data":"d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895"} Nov 04 05:03:37.776521 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.776470 4374 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" Nov 04 05:03:37.776521 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.776491 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m" event={"ID":"81b1e71b-fa07-4881-824a-8b02f19bc2eb","Type":"ContainerDied","Data":"16c1c0eab665a3dbebbcf4bba8a2364ef7516457d4245496f163a6501667ace0"} Nov 04 05:03:37.798608 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.798585 4374 scope.go:122] "RemoveContainer" containerID="16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1" Nov 04 05:03:37.798976 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:03:37.798952 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1\": container with ID starting with 16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1 not found: ID does not exist" containerID="16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1" Nov 04 05:03:37.799086 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.798985 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1"} err="failed to get container status \"16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1\": rpc error: code = NotFound desc = could not find container \"16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1\": container with ID starting with 16d19207b2ddf61a95483dde949c2d808dcbe829ba4b84b48b4f612fe6cdc2c1 not found: ID does not exist" Nov 04 05:03:37.799086 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.799003 4374 scope.go:122] "RemoveContainer" containerID="5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746" Nov 04 05:03:37.807219 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.807199 4374 scope.go:122] "RemoveContainer" containerID="5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746" Nov 04 05:03:37.807484 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:03:37.807468 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746\": container with ID starting with 5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746 not found: ID does not exist" containerID="5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746" Nov 04 05:03:37.807549 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.807490 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746"} err="failed to get container status \"5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746\": rpc error: code = NotFound desc = could not find container \"5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746\": container with ID starting with 5d0ddb27a59c20602fc4e53c357836b33b3cacc38ab1b03e828bc686d1447746 not found: ID does not exist" Nov 04 05:03:37.807549 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.807506 4374 scope.go:122] "RemoveContainer" containerID="d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895" Nov 04 05:03:37.814770 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.814754 4374 scope.go:122] "RemoveContainer" containerID="d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895" Nov 04 05:03:37.815035 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:03:37.815018 4374 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895\": container with ID starting with d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895 not found: ID does not exist" containerID="d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895" Nov 04 05:03:37.815104 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.815040 4374 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895"} err="failed to get container status \"d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895\": rpc error: code = NotFound desc = could not find container \"d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895\": container with ID starting with d9d0c472e65c83729d795e579a7fdf566216a4a5e3dee60e748a1f7717bb6895 not found: ID does not exist" Nov 04 05:03:37.926944 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.926912 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2"] Nov 04 05:03:37.973565 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:37.973539 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-7tkr2"] Nov 04 05:03:38.034155 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:38.034075 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m"] Nov 04 05:03:38.037916 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:38.037892 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-vmr9m"] Nov 04 05:03:38.141288 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:38.141261 4374 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw"] Nov 04 05:03:38.147382 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:38.147355 4374 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-a48f7e-openshift-default-b5476c79-b5tbw"] Nov 04 05:03:38.411054 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:38.410977 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51da6320-a68e-43f4-8a66-feff5075e34b" path="/var/lib/kubelet/pods/51da6320-a68e-43f4-8a66-feff5075e34b/volumes" Nov 04 05:03:38.411430 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:38.411416 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" path="/var/lib/kubelet/pods/81b1e71b-fa07-4881-824a-8b02f19bc2eb/volumes" Nov 04 05:03:38.411810 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:38.411796 4374 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" path="/var/lib/kubelet/pods/ea7cdff7-8f6a-420a-9b03-b3044df5fd28/volumes" Nov 04 05:03:44.862480 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:44.862428 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:03:45.425775 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:45.425742 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7c679bcc4f-wjccp_cfa90fbe-9a23-4420-be9b-5fee96dcdd44/manager/0.log" Nov 04 05:03:46.196441 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:46.196407 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-mxvsk_44df49b9-c08f-4cc7-8d90-fe3859a7585d/manager/0.log" Nov 04 05:03:48.931933 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:48.931901 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-597b465c59-jhv84_72c4479b-c93d-40d3-b479-3ea6c557515d/maas-api/0.log" Nov 04 05:03:55.832321 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:03:55.832284 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 05:03:56.169458 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:56.169381 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-r5g6w_4a069916-bc79-42d8-8b30-be79dde85d61/limitador/0.log" Nov 04 05:03:57.603672 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:57.603637 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-cwxsk_e7b15653-232e-4230-b0db-0db35ede9f6f/discovery/0.log" Nov 04 05:03:57.960109 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:57.960022 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-655d896fb8-b9v6x_321d39ea-e4bc-4cdf-9cf6-015d7c191dcc/kube-auth-proxy/0.log" Nov 04 05:03:58.623792 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:58.623772 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-psx2w_dc3bdc4d-9167-4b54-94a5-a364b1deb668/istio-proxy/0.log" Nov 04 05:03:59.092246 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:59.092220 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-7cvjx_e1b03de5-92e0-4f77-9548-29ba4017ef0d/payload-pre-processing/0.log" Nov 04 05:03:59.289733 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:03:59.289702 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-65r6s_01b3ed18-6bc4-41c8-bad8-bc93b08b3746/payload-processing/0.log" Nov 04 05:04:01.064975 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:01.064946 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-8txv8_1b80b52d-3ca9-4459-8ee7-90ce72f98bcf/main/0.log" Nov 04 05:04:01.336447 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:01.336363 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-rxz69_d6befc7e-171a-468b-a6b1-55c469a0cd6e/main/0.log" Nov 04 05:04:01.552384 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:01.552358 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2_4903fa82-ba60-4ad8-9a97-622f6e6361f0/main/0.log" Nov 04 05:04:02.004545 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:02.004515 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh_b023d5dd-1b1e-4c6e-8124-7fc49137d601/main/0.log" Nov 04 05:04:02.266589 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:02.266518 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw_9f3cd901-b98c-4232-8604-87ef595b35e2/main/0.log" Nov 04 05:04:12.583380 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:12.583347 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7c679bcc4f-wjccp_cfa90fbe-9a23-4420-be9b-5fee96dcdd44/manager/0.log" Nov 04 05:04:13.181912 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:13.181882 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-mxvsk_44df49b9-c08f-4cc7-8d90-fe3859a7585d/manager/0.log" Nov 04 05:04:15.253551 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:15.253520 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-597b465c59-jhv84_72c4479b-c93d-40d3-b479-3ea6c557515d/maas-api/0.log" Nov 04 05:04:21.474975 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:21.474949 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-r5g6w_4a069916-bc79-42d8-8b30-be79dde85d61/limitador/0.log" Nov 04 05:04:22.527552 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:22.527521 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-cwxsk_e7b15653-232e-4230-b0db-0db35ede9f6f/discovery/0.log" Nov 04 05:04:22.913656 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:22.913582 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-655d896fb8-b9v6x_321d39ea-e4bc-4cdf-9cf6-015d7c191dcc/kube-auth-proxy/0.log" Nov 04 05:04:23.598032 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:23.597998 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-57f78bcd55-psx2w_dc3bdc4d-9167-4b54-94a5-a364b1deb668/istio-proxy/0.log" Nov 04 05:04:24.097814 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:24.097785 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-7cvjx_e1b03de5-92e0-4f77-9548-29ba4017ef0d/payload-pre-processing/0.log" Nov 04 05:04:24.321904 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:24.321864 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-65r6s_01b3ed18-6bc4-41c8-bad8-bc93b08b3746/payload-processing/0.log" Nov 04 05:04:25.655055 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:25.655026 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-8txv8_1b80b52d-3ca9-4459-8ee7-90ce72f98bcf/main/0.log" Nov 04 05:04:25.832271 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:04:25.832238 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 05:04:25.928595 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:25.928522 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-rxz69_d6befc7e-171a-468b-a6b1-55c469a0cd6e/main/0.log" Nov 04 05:04:26.153941 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:26.153915 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dmgp2_4903fa82-ba60-4ad8-9a97-622f6e6361f0/main/0.log" Nov 04 05:04:26.621391 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:26.621366 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-5pxnh_b023d5dd-1b1e-4c6e-8124-7fc49137d601/main/0.log" Nov 04 05:04:26.854555 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:26.854524 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-6qspw_9f3cd901-b98c-4232-8604-87ef595b35e2/main/0.log" Nov 04 05:04:45.189908 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:45.189816 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:04:50.957716 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:50.957686 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-65r6s_01b3ed18-6bc4-41c8-bad8-bc93b08b3746/payload-processing/0.log" Nov 04 05:04:51.176960 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:04:51.176928 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-65r6s_01b3ed18-6bc4-41c8-bad8-bc93b08b3746/payload-processing/0.log" Nov 04 05:04:55.832339 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:04:55.832305 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 05:05:21.719538 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:21.719504 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:05:24.756643 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:24.756609 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-65r6s_01b3ed18-6bc4-41c8-bad8-bc93b08b3746/payload-processing/0.log" Nov 04 05:05:24.979764 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:24.979734 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-65r6s_01b3ed18-6bc4-41c8-bad8-bc93b08b3746/payload-processing/0.log" Nov 04 05:05:25.315587 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:25.315554 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:05:25.461650 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:25.461619 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:05:25.832150 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:05:25.832116 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 05:05:26.058819 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:26.058785 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:05:26.203015 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:26.202932 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:05:31.232054 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:31.232021 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:05:35.585237 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:35.585211 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-stlg9_ddd38197-2fcb-4086-a376-b5ef4628be3c/global-pull-secret-syncer/0.log" Nov 04 05:05:37.182605 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:37.182570 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:05:37.326735 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:37.326697 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:05:37.533422 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:37.533390 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" Nov 04 05:05:39.698894 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:39.698841 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-242.us-west-2.compute.internal_85dfaf93b35e741480ab992c223ae102/haproxy/0.log" Nov 04 05:05:44.397415 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.397372 4374 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz"] Nov 04 05:05:44.397994 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.397974 4374 cpu_manager.go:414] "containerMap: removing container" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerName="istio-proxy" Nov 04 05:05:44.398084 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.397995 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerName="istio-proxy" Nov 04 05:05:44.398084 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.398012 4374 cpu_manager.go:414] "containerMap: removing container" podUID="ad8b2336-652f-4e73-ab74-4a6c69aa5bc5" containerName="maas-api" Nov 04 05:05:44.398084 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.398020 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad8b2336-652f-4e73-ab74-4a6c69aa5bc5" containerName="maas-api" Nov 04 05:05:44.398084 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.398064 4374 cpu_manager.go:414] "containerMap: removing container" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerName="istio-proxy" Nov 04 05:05:44.398084 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.398072 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerName="istio-proxy" Nov 04 05:05:44.398332 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.398087 4374 cpu_manager.go:414] "containerMap: removing container" podUID="51da6320-a68e-43f4-8a66-feff5075e34b" containerName="istio-proxy" Nov 04 05:05:44.398332 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.398095 4374 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51da6320-a68e-43f4-8a66-feff5075e34b" containerName="istio-proxy" Nov 04 05:05:44.398332 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.398184 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea7cdff7-8f6a-420a-9b03-b3044df5fd28" containerName="istio-proxy" Nov 04 05:05:44.398332 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.398198 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad8b2336-652f-4e73-ab74-4a6c69aa5bc5" containerName="maas-api" Nov 04 05:05:44.398332 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.398213 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="81b1e71b-fa07-4881-824a-8b02f19bc2eb" containerName="istio-proxy" Nov 04 05:05:44.398332 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.398223 4374 memory_manager.go:365] "RemoveStaleState removing state" podUID="51da6320-a68e-43f4-8a66-feff5075e34b" containerName="istio-proxy" Nov 04 05:05:44.401472 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.401449 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.424580 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.424548 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6vpb\" (UniqueName: \"kubernetes.io/projected/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-kube-api-access-z6vpb\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.424718 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.424616 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-sys\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.424718 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.424695 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-lib-modules\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.424837 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.424726 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-podres\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.424918 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.424858 4374 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-proc\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.490704 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.490674 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz"] Nov 04 05:05:44.526028 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.525985 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-proc\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.526213 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.526070 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6vpb\" (UniqueName: \"kubernetes.io/projected/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-kube-api-access-z6vpb\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.526213 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.526106 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-proc\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.526213 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.526113 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-sys\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.526213 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.526148 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-lib-modules\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.526213 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.526170 4374 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-podres\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.526213 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.526183 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-sys\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.526443 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.526253 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-podres\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.526443 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.526254 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-lib-modules\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.541188 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.541163 4374 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f7wrj\"/\"default-dockercfg-kt2w6\"" Nov 04 05:05:44.553301 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.553277 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f7wrj\"/\"openshift-service-ca.crt\"" Nov 04 05:05:44.553434 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.553309 4374 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f7wrj\"/\"kube-root-ca.crt\"" Nov 04 05:05:44.583820 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.583790 4374 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6vpb\" (UniqueName: \"kubernetes.io/projected/ba0a59a1-ce2d-4816-ab30-8094582a9eb4-kube-api-access-z6vpb\") pod \"perf-node-gather-daemonset-cs5sz\" (UID: \"ba0a59a1-ce2d-4816-ab30-8094582a9eb4\") " pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:44.714641 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:44.714550 4374 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:45.079841 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:45.079789 4374 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz"] Nov 04 05:05:45.082442 ip-10-0-140-242 kubenswrapper[4374]: W1104 05:05:45.082410 4374 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podba0a59a1_ce2d_4816_ab30_8094582a9eb4.slice/crio-fc90634b926a6db9e06c32336f876560cf06e0dbfbe96c6a13cb80ba09aa5606 WatchSource:0}: Error finding container fc90634b926a6db9e06c32336f876560cf06e0dbfbe96c6a13cb80ba09aa5606: Status 404 returned error can't find the container with id fc90634b926a6db9e06c32336f876560cf06e0dbfbe96c6a13cb80ba09aa5606 Nov 04 05:05:45.356421 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:45.356337 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" event={"ID":"ba0a59a1-ce2d-4816-ab30-8094582a9eb4","Type":"ContainerStarted","Data":"c94fc7225f396d33e1088e8917456ade84c9a5f91f12006cc8e94058807c2c8f"} Nov 04 05:05:45.356421 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:45.356384 4374 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" event={"ID":"ba0a59a1-ce2d-4816-ab30-8094582a9eb4","Type":"ContainerStarted","Data":"fc90634b926a6db9e06c32336f876560cf06e0dbfbe96c6a13cb80ba09aa5606"} Nov 04 05:05:45.356421 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:45.356415 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:45.465830 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:45.465771 4374 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" podStartSLOduration=1.465756577 podStartE2EDuration="1.465756577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:05:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:05:45.464787668 +0000 UTC m=+1187.176478807" watchObservedRunningTime="2026-11-04 05:05:45.465756577 +0000 UTC m=+1187.177447716" Nov 04 05:05:45.535002 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:45.534971 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:05:46.207887 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:46.207847 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-r5g6w_4a069916-bc79-42d8-8b30-be79dde85d61/limitador/0.log" Nov 04 05:05:49.240754 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:49.240711 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ba467c78-8d75-4a2d-983d-427b4d983cca/alertmanager/0.log" Nov 04 05:05:49.646527 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:49.646426 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ba467c78-8d75-4a2d-983d-427b4d983cca/config-reloader/0.log" Nov 04 05:05:50.051112 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:50.051082 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ba467c78-8d75-4a2d-983d-427b4d983cca/kube-rbac-proxy-web/0.log" Nov 04 05:05:50.455694 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:50.455617 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ba467c78-8d75-4a2d-983d-427b4d983cca/kube-rbac-proxy/0.log" Nov 04 05:05:50.861098 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:50.861069 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ba467c78-8d75-4a2d-983d-427b4d983cca/kube-rbac-proxy-metric/0.log" Nov 04 05:05:51.265781 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:51.265751 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ba467c78-8d75-4a2d-983d-427b4d983cca/prom-label-proxy/0.log" Nov 04 05:05:51.371521 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:51.371497 4374 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f7wrj/perf-node-gather-daemonset-cs5sz" Nov 04 05:05:51.671332 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:51.671262 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ba467c78-8d75-4a2d-983d-427b4d983cca/init-config-reloader/0.log" Nov 04 05:05:52.509405 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:52.509377 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-2nggf_853f9b0c-b2ed-4f8a-9d27-1066ac8063a3/kube-state-metrics/0.log" Nov 04 05:05:52.947473 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:52.947401 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-2nggf_853f9b0c-b2ed-4f8a-9d27-1066ac8063a3/kube-rbac-proxy-main/0.log" Nov 04 05:05:53.353553 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:53.353527 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-2nggf_853f9b0c-b2ed-4f8a-9d27-1066ac8063a3/kube-rbac-proxy-self/0.log" Nov 04 05:05:53.772053 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:53.772031 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-bfb564b7d-p54wf_0d1cc979-29c1-410d-b9a7-f339222dd71c/metrics-server/0.log" Nov 04 05:05:55.832072 ip-10-0-140-242 kubenswrapper[4374]: E1104 05:05:55.832036 4374 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gt8km" containerName="extractor" Nov 04 05:05:55.869054 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:55.869029 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kvd59_7ccb6ee6-7eff-44c5-a207-a5ab97892976/node-exporter/0.log" Nov 04 05:05:56.277238 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:56.277199 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kvd59_7ccb6ee6-7eff-44c5-a207-a5ab97892976/kube-rbac-proxy/0.log" Nov 04 05:05:56.685975 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:56.685902 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kvd59_7ccb6ee6-7eff-44c5-a207-a5ab97892976/init-textfile/0.log" Nov 04 05:05:58.153806 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:58.153775 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:05:58.298784 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:58.298754 4374 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zd4x_ed101df4-83a6-42e0-8638-8b99890a8553/konnectivity-agent/0.log" limit=1024 Nov 04 05:05:58.352120 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:58.352093 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-4hdx6_bede814c-5c34-409e-a795-1cc50f629257/kube-rbac-proxy-main/0.log" Nov 04 05:05:58.756982 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:58.756949 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-4hdx6_bede814c-5c34-409e-a795-1cc50f629257/kube-rbac-proxy-self/0.log" Nov 04 05:05:59.161855 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:05:59.161775 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-4hdx6_bede814c-5c34-409e-a795-1cc50f629257/openshift-state-metrics/0.log" Nov 04 05:06:03.897172 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:06:03.897143 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-855956456c-pwkjc_2d13baed-8251-49ef-bcd6-8818db2c6b22/telemeter-client/0.log" Nov 04 05:06:04.302023 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:06:04.301996 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-855956456c-pwkjc_2d13baed-8251-49ef-bcd6-8818db2c6b22/reload/0.log" Nov 04 05:06:04.707891 ip-10-0-140-242 kubenswrapper[4374]: I1104 05:06:04.707785 4374 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-855956456c-pwkjc_2d13baed-8251-49ef-bcd6-8818db2c6b22/kube-rbac-proxy/0.log"