Nov 04 03:14:07.832454 ip-10-0-133-201 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 04 03:14:07.832467 ip-10-0-133-201 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 04 03:14:07.832477 ip-10-0-133-201 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 04 03:14:07.832797 ip-10-0-133-201 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 04 03:14:17.942584 ip-10-0-133-201 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 04 03:14:17.942603 ip-10-0-133-201 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 3249a06bf6ac4685b117019264acedc7 --
Nov 04 03:16:27.579237 ip-10-0-133-201 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 03:16:28.055221 ip-10-0-133-201 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:16:28.055221 ip-10-0-133-201 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 03:16:28.055221 ip-10-0-133-201 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:16:28.055221 ip-10-0-133-201 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:16:28.059437 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059359    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:28.059437 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059438    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059442    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059445    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059448    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059451    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059454    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059457    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059459    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059462    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059465    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059468    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059471    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059473    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059476    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059479    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059482    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059484    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059487    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059490    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:28.059508 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059492    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059495    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059498    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059500    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059503    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059505    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059516    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059519    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059522    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059525    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059527    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059530    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059533    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059535    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059538    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059541    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059543    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059545    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059548    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059551    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:28.059965 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059554    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059556    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059561    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059565    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059569    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059572    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059575    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059578    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059581    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059584    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059586    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059589    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059592    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059594    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059598    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059601    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059603    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059606    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059609    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:28.060440 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059611    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059614    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059616    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059619    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059621    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059624    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059627    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059635    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059639    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059642    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059645    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059647    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059650    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059653    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059655    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059658    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059661    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059664    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059666    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:28.060887 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059669    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059672    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059674    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059677    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059679    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059682    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059684    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059687    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059690    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059696    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059699    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059701    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059704    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059706    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059709    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059711    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059714    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059716    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059720    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059722    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:28.061338 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059725    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059727    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059730    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059732    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059735    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059737    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059740    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.059742    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061425    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061434    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061437    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061439    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061442    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061447    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061451    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061455    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061458    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061460    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061463    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:28.061812 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061466    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061468    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061471    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061479    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061482    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061485    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061487    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061490    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061492    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061495    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061498    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061502    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061505    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061508    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061511    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061514    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061517    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061520    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061523    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:28.062284 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061526    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061528    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061531    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061533    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061536    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061539    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061541    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061544    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061546    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061548    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061551    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061554    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061556    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061559    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061561    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061564    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061566    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061574    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061578    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061580    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:28.062750 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061584    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061586    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061588    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061591    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061594    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061596    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061599    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061601    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061605    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061608    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061610    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061613    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061616    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061618    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061621    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061624    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061626    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061629    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061631    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061634    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:28.063238 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061645    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061648    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061651    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061654    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061657    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061659    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061662    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061665    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061667    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061670    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061680    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061682    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061685    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061688    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061691    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061693    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061696    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061698    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061701    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061704    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:28.063722 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061707    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061710    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061712    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061715    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061717    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061720    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061722    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061725    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061727    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061730    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061732    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061735    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061738    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061741    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061743    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.061746    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062341    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062347    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062350    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:28.064226 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062353    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062356    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062358    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062367    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062370    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062373    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062375    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062378    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062381    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062383    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062386    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062389    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062391    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062394    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062397    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062399    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062402    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062405    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062407    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062410    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:28.064699 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062412    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062415    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062423    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062426    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062428    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062430    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062433    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062435    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062438    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062440    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062443    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062445    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062448    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062451    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062455    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062459    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062469    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062473    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062475    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062478    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:28.065262 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062481    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062483    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062485    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062488    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062490    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062493    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062496    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062498    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062501    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062503    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062506    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062508    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062511    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062513    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062517    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062521    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062524    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062527    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062530    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:28.065741 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062532    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062535    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062537    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062540    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062543    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062546    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062549    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062552    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062554    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062557    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062565    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062568    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062571    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062573    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062576    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062578    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062581    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062583    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062586    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062588    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:28.066255 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062591    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062594    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062596    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062599    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062601    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062604    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062606    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062609    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062611    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062614    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062616    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062619    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062624    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062627    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062629    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062632    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062634    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062637    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062640    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:28.066729 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062642    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062645    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062647    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062650    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.062655    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063763    2574 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063772    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063780    2574 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063785    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063790    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063793    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063798    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063802    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063806    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063809    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063812    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063816    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063819    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063822    2574 flags.go:64] FLAG: --cgroup-root=""
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063825    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063828    2574 flags.go:64] FLAG: --client-ca-file=""
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063831    2574 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063834    2574 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 03:16:28.067184 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063838    2574 flags.go:64] FLAG: --cluster-domain=""
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063841    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063844    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063847    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063850    2574 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063855    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063858    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063861    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063864    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063867    2574 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063870    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063873    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063876    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063879    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063890    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063893    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063896    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063899    2574 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063902    2574 flags.go:64] FLAG: --enable-server="true"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063919    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063930    2574 flags.go:64] FLAG: --event-burst="100"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063935    2574 flags.go:64] FLAG: --event-qps="50"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063938    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063942    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 03:16:28.067756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063944    2574 flags.go:64] FLAG: --eviction-hard=""
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063949    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063952    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063955    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063958    2574 flags.go:64] FLAG: --eviction-soft=""
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063961    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063964    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063967    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063970    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063973    2574 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063976    2574 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063979    2574 flags.go:64] FLAG: --feature-gates=""
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063984    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063987    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063990    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063993    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.063996    2574 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064000    2574 flags.go:64] FLAG: --help="false"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064002    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064006    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064009    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064011    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064015    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 03:16:28.068368 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064018    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064021    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064024    2574 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064027    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064030    2574 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064033    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064036    2574 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064039    2574 flags.go:64] FLAG: --kube-reserved=""
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064042    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064044    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064047    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064050    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064053    2574 flags.go:64] FLAG: --lock-file=""
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064056    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064059    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064062    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064067    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064070    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064073    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064076    2574 flags.go:64] FLAG: --logging-format="text"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064079    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064082    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064085    2574 flags.go:64] FLAG: --manifest-url=""
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064088    2574 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064095    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 03:16:28.068955 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064098    2574 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064102    2574 flags.go:64] FLAG: --max-pods="110"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064106    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064108    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064111    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064114    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064117    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064120    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064129    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064136    2574 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064139    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064142    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064146    2574 flags.go:64] FLAG: --pod-cidr=""
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064149    2574 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064152    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064155    2574 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064157    2574 flags.go:64] FLAG: --port="10250"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064161    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064163    2574 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-051361a2477c1b821"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064167    2574 flags.go:64] FLAG: --qos-reserved=""
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064169    2574 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064172    2574 flags.go:64] FLAG: --register-node="true"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064175    2574 flags.go:64] FLAG: --register-with-taints=""
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064179    2574 flags.go:64] FLAG: --registry-burst="10"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064182    2574 flags.go:64] FLAG: --registry-qps="5"
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064185    2574 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 03:16:28.069544 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064187    2574 flags.go:64] FLAG: --reserved-memory=""
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064191    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064194    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064197    2574 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064200    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064203    2574 flags.go:64] FLAG: --runonce="false"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064206    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064209    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064212    2574 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064215    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064218    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064221    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064224    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064227    2574 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064231    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064235    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064239    2574 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064242    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064245    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064248    2574 flags.go:64] FLAG: --system-cgroups=""
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064251    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064257    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064259    2574 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064262    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064269    2574 flags.go:64] FLAG: --tls-min-version=""
Nov 04 03:16:28.070208 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064274    2574 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064277    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064280    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064283    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064286    2574 flags.go:64] FLAG: --v="2"
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064290    2574 flags.go:64] FLAG: --version="false"
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064295    2574 flags.go:64] FLAG: --vmodule=""
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064299    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064302    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064493    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064498    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064502    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064506    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064509    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064511    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064514    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064517    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064519    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064522    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064524    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064527    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:28.070854 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064529    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064532    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064543    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064546    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064548    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064551    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064554    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064556    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064559    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064561    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064564    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064569    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064572    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064575    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064578    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064581    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064583    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064586    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064588    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:28.071361 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064591    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064594    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064596    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064599    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064601    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064604    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064607    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064609    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064612    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064614    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064617    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064619    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064622    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064625    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064627    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064632    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064640    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064642    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064647    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064650    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:28.071799 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064653    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064655    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064658    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064661    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064664    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064667    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064669    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064672    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064674    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064677    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064680    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064682    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064685    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064688    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064690    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064693    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064695    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064698    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064701    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064704    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:28.072290 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064706    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064709    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064712    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064715    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064723    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064726    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064728    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064732    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064735    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064743    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064746    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064748    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064751    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064754    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064756    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064759    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064763    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064766    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064769    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:28.072762 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064771    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064774    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064776    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064779    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064781    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064784    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064787    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064789    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064791    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064794    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064797    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064799    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064802    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064804    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064807    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064809    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:28.073227 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.064836    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064877    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064882    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064885    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064889    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064892    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064895    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064903    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064919    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064922    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064925    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064927    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064930    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064933    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064939    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064942    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064945    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064948    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064950    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064953    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:28.073610 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064955    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064958    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064960    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064963    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064966    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064968    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064971    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064974    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064976    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064979    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064981    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064984    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064987    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064989    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064992    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064994    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.064998    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065000    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065003    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065006    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:28.074123 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065015    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065018    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065022    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065027    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065031    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065035    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065040    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065044    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065048    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065051    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065054    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065057    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065059    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065062    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065065    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065067    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065070    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065072    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065075    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:28.074794 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065078    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065080    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065083    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065085    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065088    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065090    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065093    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065096    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065098    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065102    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065104    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065107    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065111    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065115    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065125    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065128    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065130    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065133    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065136    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:28.075480 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065139    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065142    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065144    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065147    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065150    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065152    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065155    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065157    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065160    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065163    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065165    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065168    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065171    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065173    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065176    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065178    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065181    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065183    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065186    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:28.075972 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065188    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065191    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065193    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065197    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065200    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065202    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065205    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065208    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065210    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.065218    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.071994    2574 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.072012    2574 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.072028    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.072034    2574 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072088    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072093    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072097    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072101    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072104    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:28.076431 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072107    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072110    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072113    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072115    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072118    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072121    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072125    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072129    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072133    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072137    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072141    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072144    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072147    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072150    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072152    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072155    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072157    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072160    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072163    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:28.076903 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072166    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072168    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072171    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072173    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072176    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072180    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072183    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072186    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072189    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072191    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072194    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072196    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072199    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072202    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072204    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072207    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072210    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072212    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072215    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072217    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:28.077363 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072220    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072222    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072225    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072227    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072230    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072233    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072235    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072238    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072241    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072243    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072246    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072248    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072251    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072253    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072256    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072258    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072261    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072265    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072268    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072271    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:28.077841 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072273    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072276    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072278    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072281    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072284    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072286    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072289    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072292    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072294    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072297    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072300    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072302    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072305    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072307    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072310    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072312    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072315    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072318    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072320    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072323    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:28.078426 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072325    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072328    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072331    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072333    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072335    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072338    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072341    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072343    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072346    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072349    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072352    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072355    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072358    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072360    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072363    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072365    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072368    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072371    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072373    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:28.078882 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072376    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072379    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072382    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.072409    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072543    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072547    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072550    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072553    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072555    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072558    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072561    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072563    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072566    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072570    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072574    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:28.079334 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072578    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072581    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072583    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072586    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072589    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072592    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072594    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072597    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072601    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072603    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072606    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072609    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072612    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072615    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072617    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072620    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072622    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072625    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072627    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:28.079678 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072630    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072632    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072635    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072638    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072640    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072642    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072645    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072647    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072650    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072653    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072655    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072658    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072660    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072663    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072665    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072668    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072670    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072673    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072675    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072678    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:28.080180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072681    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072684    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072686    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072689    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072696    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072699    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072701    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072704    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072707    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072709    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072712    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072714    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072717    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072719    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072722    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072724    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072727    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072729    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072732    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072735    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:28.080670 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072737    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072740    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072742    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072745    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072747    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072750    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072752    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072755    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072757    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072760    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072763    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072766    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072769    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072772    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072774    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072777    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072779    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072783    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072786    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072788    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:28.081159 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072791    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072793    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072796    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072799    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072801    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072803    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072806    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072809    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072812    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072815    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072819    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072822    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072825    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072828    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072830    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:28.081620 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072833    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.072855    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072886    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072890    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072893    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072896    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072898    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072901    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072904    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072921    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072925    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072928    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072930    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072933    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072935    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072939    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:16:28.082040 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072942    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072944    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072947    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072949    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072952    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072955    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072957    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072960    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072963    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072965    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072968    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072970    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072973    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072975    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072978    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072980    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072983    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072986    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072988    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072991    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:16:28.082420 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072993    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072996    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.072998    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073001    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073004    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073007    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073010    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073012    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073015    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073017    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073020    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073022    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073025    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073028    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073030    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073033    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073035    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073038    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073040    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:16:28.082877 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073043    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073046    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073048    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073051    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073053    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073056    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073058    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073061    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073063    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073066    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073068    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073071    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073073    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073075    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073078    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073080    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073083    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073086    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073089    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073091    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:16:28.083487 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073095    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073099    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073102    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073105    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073107    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073109    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073112    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073115    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073117    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073120    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073122    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073125    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073128    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073130    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073133    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073135    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073138    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073141    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073145    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:16:28.083987 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073148    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073151    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073153    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073156    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073158    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073161    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073163    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073166    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073168    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073171    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073173    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073177    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073180    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:28.073182    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.073250    2574 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.077351    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.078402    2574 server.go:1014] "Starting client certificate rotation"
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.078494    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 03:16:28.084531 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.078524    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 03:16:28.084978 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.080616    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:16:28.084978 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.081852    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:16:28.090826 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.090813    2574 log.go:25] "Validated CRI v1 runtime API"
Nov 04 03:16:28.094308 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.094295    2574 log.go:25] "Validated CRI v1 image API"
Nov 04 03:16:28.096106 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.096075    2574 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 03:16:28.100562 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.100538    2574 fs.go:135] Filesystem UUIDs: map[6ca3d19c-ee97-4728-8a18-cc6b7f7ef1b5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a6a02d61-27a8-4448-bcfa-8fc8c9ef354a:/dev/nvme0n1p3]
Nov 04 03:16:28.100645 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.100559    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 03:16:28.105495 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.105387    2574 manager.go:217] Machine: {Timestamp:2026-11-04 03:16:28.104070254 +0000 UTC m=+0.389503541 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3077224 MemoryCapacity:33164328960 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a56e9b9ee18983e02fc8c56b6dd47 SystemUUID:ec2a56e9-b9ee-1898-3e02-fc8c56b6dd47 BootID:3249a06b-f6ac-4685-b117-019264acedc7 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:e2:99:98:43 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:e2:99:98:43 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:47:6b:c9:05:3a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164328960 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 03:16:28.105570 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.105496    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 03:16:28.105605 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.105594    2574 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 03:16:28.106810 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.106784    2574 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 03:16:28.106954 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.106812    2574 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-201.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 03:16:28.107002 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.106956    2574 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 03:16:28.107002 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.106965    2574 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 03:16:28.107002 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.106994    2574 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:16:28.107080 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.107010    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:16:28.107976 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.107966    2574 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 03:16:28.112287 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.112275    2574 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 03:16:28.112420 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.112410    2574 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 03:16:28.113747 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.113737    2574 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 03:16:28.113779 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.113751    2574 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 03:16:28.113779 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.113761    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 03:16:28.113779 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.113770    2574 kubelet.go:405] "Adding apiserver pod source"
Nov 04 03:16:28.113779 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.113779    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 03:16:28.114204 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.114190    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:16:28.119462 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.119392    2574 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 03:16:28.124785 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.124766    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:16:28.125903 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.125889    2574 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 03:16:28.125962 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.125935    2574 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 03:16:28.126095 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.126084    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 03:16:28.126128 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.126100    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 03:16:28.126128 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.126106    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 03:16:28.126128 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.126112    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 03:16:28.126128 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.126118    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 03:16:28.126128 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.126124    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 03:16:28.126778 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.126770    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 03:16:28.126778 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.126778    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 03:16:28.126830 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.126785    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 03:16:28.126830 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.126791    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 03:16:28.126830 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.126800    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 03:16:28.126830 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.126809    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 03:16:28.128150 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.128140    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 03:16:28.128150 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.128149    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 03:16:28.131881 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.131868    2574 server.go:1263] "Started kubelet"
Nov 04 03:16:28.131990 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.131944    2574 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 03:16:28.132138 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.131957    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 03:16:28.132229 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.132161    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 03:16:28.132556 ip-10-0-133-201 systemd[1]: Started Kubernetes Kubelet.
Nov 04 03:16:28.133474 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.133457    2574 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 03:16:28.136153 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.136141    2574 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 03:16:28.139650 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.139631    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 03:16:28.139803 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.139782    2574 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 03:16:28.140776 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.139970    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:28.140776 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.140412    2574 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 03:16:28.140942 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.140929    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 03:16:28.140999 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.140932    2574 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 03:16:28.140999 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.140956    2574 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 03:16:28.141081 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.141057    2574 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 03:16:28.141081 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.141068    2574 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 03:16:28.141530 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.141513    2574 factory.go:55] Registering systemd factory
Nov 04 03:16:28.141580 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.141533    2574 factory.go:223] Registration of the systemd container factory successfully
Nov 04 03:16:28.141749 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.141735    2574 factory.go:153] Registering CRI-O factory
Nov 04 03:16:28.141804 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.141752    2574 factory.go:223] Registration of the crio container factory successfully
Nov 04 03:16:28.141853 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.141828    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 03:16:28.141901 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.141859    2574 factory.go:103] Registering Raw factory
Nov 04 03:16:28.141901 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.141875    2574 manager.go:1199] Started watching for new ooms in manager
Nov 04 03:16:28.142879 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.142865    2574 manager.go:319] Starting recovery of all containers
Nov 04 03:16:28.151023 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.150970    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 03:16:28.158215 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.158197    2574 manager.go:324] Recovery completed
Nov 04 03:16:28.164207 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.164193    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:28.165991 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.165970    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:28.166084 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.166009    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:28.166084 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.166036    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:28.166609 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.166585    2574 cpu_manager.go:226] "Starting" policy="none"
Nov 04 03:16:28.166609 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.166597    2574 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 03:16:28.166748 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.166613    2574 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 03:16:28.166748 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.166588    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:28.168376 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.168360    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:28.168429 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.168388    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:28.168429 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.168398    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:28.168701 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.168688    2574 policy_none.go:50] "Start"
Nov 04 03:16:28.168748 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.168706    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 03:16:28.168748 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.168719    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 03:16:28.170702 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.170692    2574 policy_none.go:44] "Start"
Nov 04 03:16:28.241133 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.241113    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:28.262826 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.262800    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 03:16:28.262931 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.262831    2574 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 03:16:28.262931 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.262853    2574 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 03:16:28.262931 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.262891    2574 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 03:16:28.268412 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.268389    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-201.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 03:16:28.268522 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.268390    2574 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-201.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 03:16:28.269505 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.268391    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-201.us-west-2.compute.internal.18e4b934d320385a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-201.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-201.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 03:16:28.131842138 +0000 UTC m=+0.417275425,LastTimestamp:2026-11-04 03:16:28.131842138 +0000 UTC m=+0.417275425,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-201.us-west-2.compute.internal,}"
Nov 04 03:16:28.335106 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.335029    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-201.us-west-2.compute.internal.18e4b934d5294bc5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-201.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-201.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-201.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 03:16:28.165991365 +0000 UTC m=+0.451424664,LastTimestamp:2026-11-04 03:16:28.165991365 +0000 UTC m=+0.451424664,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-201.us-west-2.compute.internal,}"
Nov 04 03:16:28.335292 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.335270    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 04 03:16:28.335418 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.335395    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-201.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 04 03:16:28.339482 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.339455    2574 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 03:16:28.341661 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.341645    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:28.341759 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.341749    2574 manager.go:341] "Starting Device Plugin manager"
Nov 04 03:16:28.341798 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.341781    2574 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 03:16:28.341798 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.341790    2574 server.go:87] "Starting device plugin registration server"
Nov 04 03:16:28.342056 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.342043    2574 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 03:16:28.342115 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.342056    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 03:16:28.342157 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.342141    2574 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 03:16:28.342231 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.342219    2574 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 03:16:28.342278 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.342234    2574 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 03:16:28.342763 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.342742    2574 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 03:16:28.342842 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.342793    2574 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:28.363457 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.363435    2574 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"]
Nov 04 03:16:28.363526 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.363508    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:28.364330 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.364316    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:28.364414 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.364346    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:28.364414 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.364360    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:28.367202 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.367187    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:28.367334 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.367320    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.367379 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.367348    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:28.367953 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.367935    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:28.368042 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.367966    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:28.368042 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.367966    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:28.368042 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.367977    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:28.368042 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.367990    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:28.368042 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.368004    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:28.370348 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.370332    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.370399 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.370367    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:28.371102 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.371086    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:28.371186 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.371112    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:28.371186 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.371125    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:28.387383 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.387365    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 03:16:28.387456 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.387401    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z6r7n"
Nov 04 03:16:28.399759 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.399744    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found" node="ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.403060 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.403044    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found" node="ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.417419 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.417341    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-201.us-west-2.compute.internal.18e4b934d529c8a9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-201.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-201.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-201.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 03:16:28.166023337 +0000 UTC m=+0.451456625,LastTimestamp:2026-11-04 03:16:28.166023337 +0000 UTC m=+0.451456625,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-201.us-west-2.compute.internal,}"
Nov 04 03:16:28.417510 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.417426    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 04 03:16:28.442742 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.442726    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:28.443436 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.443421    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:28.443501 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.443447    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:28.443501 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.443465    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:28.443501 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.443485    2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.484638 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.484617    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:28.497179 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.497163    2574 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.497241 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.497181    2574 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-201.us-west-2.compute.internal\": node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:28.541744 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.541725    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d7792c1def8a40547a57207fd144414-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"4d7792c1def8a40547a57207fd144414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.541797 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.541750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d7792c1def8a40547a57207fd144414-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"4d7792c1def8a40547a57207fd144414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.541797 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.541766    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72ddb4465d8ebfdcb6c7be795263771f-config\") pod \"kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"72ddb4465d8ebfdcb6c7be795263771f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.611135 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.611080    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:28.642788 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.642765    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d7792c1def8a40547a57207fd144414-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"4d7792c1def8a40547a57207fd144414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.642888 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.642793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d7792c1def8a40547a57207fd144414-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"4d7792c1def8a40547a57207fd144414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.642888 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.642818    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72ddb4465d8ebfdcb6c7be795263771f-config\") pod \"kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"72ddb4465d8ebfdcb6c7be795263771f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.642888 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.642849    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d7792c1def8a40547a57207fd144414-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"4d7792c1def8a40547a57207fd144414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.642888 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.642855    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d7792c1def8a40547a57207fd144414-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"4d7792c1def8a40547a57207fd144414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.643047 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.642900    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72ddb4465d8ebfdcb6c7be795263771f-config\") pod \"kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"72ddb4465d8ebfdcb6c7be795263771f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.703146 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.703125    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.706592 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:28.706578    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:28.711432 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:28.711418    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:29.915446 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:29.915419    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:30.016234 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:30.016210    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:30.116757 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:30.116731    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:30.182623 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.182602    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 03:16:30.183088 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.182763    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 03:16:30.183088 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:30.182732    2574 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://acd60e97cb07342d6a02bc86b8f04d90-7f8ad830d0f07e24.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-133-201.us-west-2.compute.internal.18e4b934d529c8a9\": read tcp 10.0.133.201:44120->3.19.105.97:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-133-201.us-west-2.compute.internal.18e4b934d529c8a9  default   5001 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-201.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-201.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-201.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 03:16:28 +0000 UTC,LastTimestamp:2026-11-04 03:16:28.367971412 +0000 UTC m=+0.653404699,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-201.us-west-2.compute.internal,}"
Nov 04 03:16:30.216921 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:30.216903    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:30.317169 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:30.317097    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:30.417746 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:30.417585    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:30.483526 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.483504    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:16:30.484388 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.484369    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:16:30.484485 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.484399    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:16:30.484485 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.484409    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:16:30.496677 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.496642    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 03:11:29 +0000 UTC" deadline="2028-05-14 02:31:33.593877887 +0000 UTC"
Nov 04 03:16:30.496677 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.496673    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13367h15m3.097207842s"
Nov 04 03:16:30.517919 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:30.517900    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:16:30.518485 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.518472    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:30.596831 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.596773    2574 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:30.645305 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.645283    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:30.713009 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.712984    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:16:30.714409 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.714396    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:16:30.772619 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:30.772600    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:16:31.013378 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.013351    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:31.018180 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:31.018152    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d7792c1def8a40547a57207fd144414.slice/crio-8dc619aefb0904ee71ece245244cf08e87f4e4a160db525930f37e2107a07489 WatchSource:0}: Error finding container 8dc619aefb0904ee71ece245244cf08e87f4e4a160db525930f37e2107a07489: Status 404 returned error can't find the container with id 8dc619aefb0904ee71ece245244cf08e87f4e4a160db525930f37e2107a07489
Nov 04 03:16:31.022667 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.022650    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:16:31.218198 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.218143    2574 apiserver.go:52] "Watching apiserver"
Nov 04 03:16:31.245234 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.245211    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 03:16:31.256733 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.256713    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:16:31.325391 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.325362    2574 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 03:16:31.325702 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.325683    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-l74dq","openshift-image-registry/node-ca-8r4z6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal","openshift-multus/network-metrics-daemon-rz4mh","openshift-multus/multus-additional-cni-plugins-s2tvj","openshift-network-diagnostics/network-check-target-5zxn5","openshift-network-operator/iptables-alerter-n6s5j","openshift-dns/node-resolver-gccvl","openshift-multus/multus-qsm2l","kube-system/konnectivity-agent-6x62l","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px","openshift-ovn-kubernetes/ovnkube-node-vg574","kube-system/global-pull-secret-syncer-tmrtb","kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"]
Nov 04 03:16:31.329323 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.329306    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:16:31.331295 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.331280    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:31.331374 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.331333    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:31.333268 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.333252    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:16:31.335383 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.335362    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:31.335468 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.335445    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:31.337468 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.337452    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.340003 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.339984    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:31.340078 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.340037    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:31.343310 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.343288    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.345433 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.345416    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.347689 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.347676    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:16:31.349928 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.349910    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:16:31.352333 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.352316    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.354659 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.354643    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.354725 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.354660    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-kubelet\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.354725 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.354693    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysctl-d\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.354838 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.354718    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-systemd\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.354838 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.354781    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-run\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.354838 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.354818    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dnnx\" (UniqueName: \"kubernetes.io/projected/e1f6fbb1-af6e-43e4-a671-68a6c3691463-kube-api-access-8dnnx\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:16:31.354979 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.354866    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-netns\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.354979 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.354893    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:31.354979 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.354954    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.355091 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.354996    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.355091 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355028    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-lib-modules\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.355091 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355046    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-var-lib-kubelet\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.355091 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355061    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9499\" (UniqueName: \"kubernetes.io/projected/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-kube-api-access-s9499\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.355206 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355096    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1f6fbb1-af6e-43e4-a671-68a6c3691463-tmp-dir\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:16:31.355206 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355121    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grkj6\" (UniqueName: \"kubernetes.io/projected/50f36ad6-52cd-4239-94eb-22b8f3de2a26-kube-api-access-grkj6\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:16:31.355206 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355165    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cni-binary-copy\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.355206 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355196    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-host\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.355319 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-tuned\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.355319 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355241    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-tmp\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.355319 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355272    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-hostroot\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.355319 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355295    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0599c4e4-3db5-42b2-bd74-2c874ab74383-serviceca\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:16:31.355436 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355325    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/39429a51-69a0-4bce-9029-773d5aad2a87-dbus\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:31.355436 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355342    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1f6fbb1-af6e-43e4-a671-68a6c3691463-hosts-file\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:16:31.355436 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355364    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-socket-dir-parent\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.355436 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355398    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.355436 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-daemon-config\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.355574 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355459    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0599c4e4-3db5-42b2-bd74-2c874ab74383-host\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:16:31.355574 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355484    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-cni-bin\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.355574 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355517    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-cni-multus\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.355574 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355543    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-modprobe-d\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.355574 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355563    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-sys\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.355720 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355589    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-system-cni-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.355720 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355611    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22q58\" (UniqueName: \"kubernetes.io/projected/e0c7e7cd-abcd-429b-b556-947ac5d9402a-kube-api-access-22q58\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.355720 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355628    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50f36ad6-52cd-4239-94eb-22b8f3de2a26-host-slash\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:16:31.355720 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355648    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysconfig\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.355720 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355704    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:31.355919 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355749    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/50f36ad6-52cd-4239-94eb-22b8f3de2a26-iptables-alerter-script\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:16:31.355919 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355786    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz7p8\" (UniqueName: \"kubernetes.io/projected/ae2e6da6-1e6d-4b54-8136-359490273f9c-kube-api-access-wz7p8\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:31.355919 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355812    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-system-cni-dir\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.355919 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355829    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5st2r\" (UniqueName: \"kubernetes.io/projected/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-kube-api-access-5st2r\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.355919 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355866    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-kubernetes\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.355919 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355902    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-conf-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.356146 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355933    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lswx\" (UniqueName: \"kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx\") pod \"network-check-target-5zxn5\" (UID: \"7566445c-b5ac-424d-aa38-a2c5b2e0de21\") " pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:31.356146 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355968    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-cni-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.356146 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.355988    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-cnibin\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.356146 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.356001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0c7e7cd-abcd-429b-b556-947ac5d9402a-cni-binary-copy\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.356146 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.356031    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-k8s-cni-cncf-io\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.356146 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.356059    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cnibin\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.356146 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.356077    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysctl-conf\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.356146 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.356107    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-multus-certs\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.356146 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.356146    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-etc-kubernetes\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.356419 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.356163    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzzhc\" (UniqueName: \"kubernetes.io/projected/0599c4e4-3db5-42b2-bd74-2c874ab74383-kube-api-access-rzzhc\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:16:31.356419 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.356190    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/39429a51-69a0-4bce-9029-773d5aad2a87-kubelet-config\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:31.372988 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.372949    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal" event={"ID":"4d7792c1def8a40547a57207fd144414","Type":"ContainerStarted","Data":"8dc619aefb0904ee71ece245244cf08e87f4e4a160db525930f37e2107a07489"}
Nov 04 03:16:31.384356 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.384339    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 03:16:31.384356 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.384347    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 03:16:31.384464 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.384342    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:16:31.384464 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.384387    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fbcnm\""
Nov 04 03:16:31.387600 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.387585    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 03:16:31.387669 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.387641    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 03:16:31.387669 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.387654    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h66cx\""
Nov 04 03:16:31.391892 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.391873    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 03:16:31.391965 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.391921    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 03:16:31.391965 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.391938    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 03:16:31.391965 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.391942    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x6xxb\""
Nov 04 03:16:31.391965 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.391956    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 03:16:31.391965 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.391962    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 03:16:31.397548 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.397534    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 03:16:31.397548 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.397546    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-569bn\""
Nov 04 03:16:31.399477 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.399463    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:16:31.399477 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.399472    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 03:16:31.399569 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.399562    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4vrb5\""
Nov 04 03:16:31.401942 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.401924    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 03:16:31.402025 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.401992    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z2nst\""
Nov 04 03:16:31.402086 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.402072    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 03:16:31.402128 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.402089    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 03:16:31.431940 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.431917    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-vpndz\""
Nov 04 03:16:31.431940 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.431936    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 03:16:31.432075 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.432027    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 03:16:31.432205 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.432190    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 03:16:31.432458 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.432444    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 03:16:31.432458 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.432456    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 03:16:31.432572 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.432467    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-56wj9\""
Nov 04 03:16:31.432572 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.432493    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 03:16:31.436520 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.436506    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 03:16:31.436570 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.436544    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9cxzk\""
Nov 04 03:16:31.436678 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.436657    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 03:16:31.436678 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.436675    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 03:16:31.436833 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.436682    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 03:16:31.436833 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.436801    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 03:16:31.436833 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.436831    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cp4xm"
Nov 04 03:16:31.436970 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.436887    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 03:16:31.446363 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.446346    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 03:16:31.456816 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.456797    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dnnx\" (UniqueName: \"kubernetes.io/projected/e1f6fbb1-af6e-43e4-a671-68a6c3691463-kube-api-access-8dnnx\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:16:31.456884 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.456822    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-netns\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.456884 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.456839    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.456950 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.456885    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-netns\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.456990 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.456959    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.457021 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.456994    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9499\" (UniqueName: \"kubernetes.io/projected/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-kube-api-access-s9499\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.457064 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457021    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-etc-selinux\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.457064 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457049    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.457159 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457045    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-cni-bin\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.457213 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457161    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cni-binary-copy\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.457269 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457210    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-tuned\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.457269 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457235    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-slash\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.457269 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457250    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39640be1-9711-4cef-a9a2-430ea40c7c18-ovn-node-metrics-cert\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.457411 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457274    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.457411 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457298    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-cni-multus\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.457411 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457333    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-ovnkube-config\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.457411 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457387    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-cni-multus\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.457634 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457407    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-system-cni-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.457634 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457447    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50f36ad6-52cd-4239-94eb-22b8f3de2a26-host-slash\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:16:31.457634 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457472    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50f36ad6-52cd-4239-94eb-22b8f3de2a26-host-slash\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:16:31.457634 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457504    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysconfig\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.457634 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457545    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:31.457634 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457559    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-system-cni-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.457634 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457582    2574 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 03:16:31.457634 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457613    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysconfig\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.457634 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457623    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457566    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz7p8\" (UniqueName: \"kubernetes.io/projected/ae2e6da6-1e6d-4b54-8136-359490273f9c-kube-api-access-wz7p8\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.457644    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457669    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-system-cni-dir\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.457697    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret podName:39429a51-69a0-4bce-9029-773d5aad2a87 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:31.95768071 +0000 UTC m=+3.139178951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret") pod "global-pull-secret-syncer-tmrtb" (UID: "39429a51-69a0-4bce-9029-773d5aad2a87") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457711    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5st2r\" (UniqueName: \"kubernetes.io/projected/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-kube-api-access-5st2r\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457728    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-kubernetes\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457734    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-system-cni-dir\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457766    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-env-overrides\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457780    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-ovnkube-script-lib\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457822    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-kubernetes\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457819    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-host\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457875    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-conf-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457878    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-host\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457901    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lswx\" (UniqueName: \"kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx\") pod \"network-check-target-5zxn5\" (UID: \"7566445c-b5ac-424d-aa38-a2c5b2e0de21\") " pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457903    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-conf-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.458057 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.457927    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-cni-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458026    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-k8s-cni-cncf-io\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458065    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-k8s-cni-cncf-io\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458071    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysctl-conf\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458088    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-cni-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458116    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-run\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458145    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458153    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-run\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458166    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysctl-conf\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-lib-modules\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458200    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-var-lib-kubelet\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458233    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-var-lib-kubelet\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458257    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3aef6eaf-2dde-4706-bcf9-77bf9c447028-agent-certs\") pod \"konnectivity-agent-6x62l\" (UID: \"3aef6eaf-2dde-4706-bcf9-77bf9c447028\") " pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458279    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzzhc\" (UniqueName: \"kubernetes.io/projected/0599c4e4-3db5-42b2-bd74-2c874ab74383-kube-api-access-rzzhc\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458295    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/39429a51-69a0-4bce-9029-773d5aad2a87-kubelet-config\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458315    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-systemd\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458340    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-device-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.458665 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458315    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-lib-modules\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458367    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-kubelet\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458368    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-systemd\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458402    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458446    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458456    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cni-binary-copy\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458467    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/39429a51-69a0-4bce-9029-773d5aad2a87-kubelet-config\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458470    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-sys-fs\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458503    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-run-netns\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.458506    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458532    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-systemd\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.458559    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs podName:ae2e6da6-1e6d-4b54-8136-359490273f9c nodeName:}" failed. No retries permitted until 2026-11-04 03:16:31.958546603 +0000 UTC m=+3.140044828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs") pod "network-metrics-daemon-rz4mh" (UID: "ae2e6da6-1e6d-4b54-8136-359490273f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458586    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-etc-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458608    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-ovn\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458630    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458651    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsm6h\" (UniqueName: \"kubernetes.io/projected/39640be1-9711-4cef-a9a2-430ea40c7c18-kube-api-access-tsm6h\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.459446 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458673    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1f6fbb1-af6e-43e4-a671-68a6c3691463-tmp-dir\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458703    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grkj6\" (UniqueName: \"kubernetes.io/projected/50f36ad6-52cd-4239-94eb-22b8f3de2a26-kube-api-access-grkj6\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458737    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl5th\" (UniqueName: \"kubernetes.io/projected/e1685aa8-41b4-4899-ad2d-6e03e2969d72-kube-api-access-bl5th\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458785    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-systemd-units\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458803    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-var-lib-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458821    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-hostroot\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458836    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0599c4e4-3db5-42b2-bd74-2c874ab74383-serviceca\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458877    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/39429a51-69a0-4bce-9029-773d5aad2a87-dbus\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458888    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-hostroot\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458901    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1f6fbb1-af6e-43e4-a671-68a6c3691463-hosts-file\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458934    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-socket-dir-parent\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458967    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3aef6eaf-2dde-4706-bcf9-77bf9c447028-konnectivity-ca\") pod \"konnectivity-agent-6x62l\" (UID: \"3aef6eaf-2dde-4706-bcf9-77bf9c447028\") " pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.458984    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-socket-dir-parent\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459012    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/39429a51-69a0-4bce-9029-773d5aad2a87-dbus\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459015    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-node-log\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459035    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1f6fbb1-af6e-43e4-a671-68a6c3691463-hosts-file\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459052    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-daemon-config\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459078    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0599c4e4-3db5-42b2-bd74-2c874ab74383-host\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:16:31.460228 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459100    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0599c4e4-3db5-42b2-bd74-2c874ab74383-host\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459102    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-cni-bin\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459123    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-cni-bin\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459140    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-modprobe-d\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459162    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-sys\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459189    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-cni-netd\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459230    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22q58\" (UniqueName: \"kubernetes.io/projected/e0c7e7cd-abcd-429b-b556-947ac5d9402a-kube-api-access-22q58\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459238    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-sys\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459261    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/50f36ad6-52cd-4239-94eb-22b8f3de2a26-iptables-alerter-script\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459262    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-modprobe-d\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459295    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-run-ovn-kubernetes\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-tmp\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459365    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-cnibin\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459408    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0c7e7cd-abcd-429b-b556-947ac5d9402a-cni-binary-copy\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459434    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cnibin\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459446    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1f6fbb1-af6e-43e4-a671-68a6c3691463-tmp-dir\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459460    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-registration-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459464    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-cnibin\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461104 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459504    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-multus-certs\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459528    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-etc-kubernetes\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459543    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cnibin\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459577    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-kubelet\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459594    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-etc-kubernetes\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459600    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysctl-d\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459621    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0599c4e4-3db5-42b2-bd74-2c874ab74383-serviceca\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459645    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-socket-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459665    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-log-socket\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459671    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-multus-certs\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459714    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-kubelet\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459785    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysctl-d\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.459791    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/50f36ad6-52cd-4239-94eb-22b8f3de2a26-iptables-alerter-script\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.460098    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-daemon-config\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.460155    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0c7e7cd-abcd-429b-b556-947ac5d9402a-cni-binary-copy\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.461258    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-tuned\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.461777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.461595    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-tmp\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.518759 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.518735    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:31.518759 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.518763    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:31.518948 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.518776    2574 projected.go:196] Error preparing data for projected volume kube-api-access-5lswx for pod openshift-network-diagnostics/network-check-target-5zxn5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:31.518948 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.518833    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx podName:7566445c-b5ac-424d-aa38-a2c5b2e0de21 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:32.018817157 +0000 UTC m=+3.200315404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5lswx" (UniqueName: "kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx") pod "network-check-target-5zxn5" (UID: "7566445c-b5ac-424d-aa38-a2c5b2e0de21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:31.521240 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.521215    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dnnx\" (UniqueName: \"kubernetes.io/projected/e1f6fbb1-af6e-43e4-a671-68a6c3691463-kube-api-access-8dnnx\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:16:31.521389 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.521373    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9499\" (UniqueName: \"kubernetes.io/projected/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-kube-api-access-s9499\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.521492 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.521473    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz7p8\" (UniqueName: \"kubernetes.io/projected/ae2e6da6-1e6d-4b54-8136-359490273f9c-kube-api-access-wz7p8\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:31.521530 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.521492    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grkj6\" (UniqueName: \"kubernetes.io/projected/50f36ad6-52cd-4239-94eb-22b8f3de2a26-kube-api-access-grkj6\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:16:31.521637 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.521213    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5st2r\" (UniqueName: \"kubernetes.io/projected/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-kube-api-access-5st2r\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.522055 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.522037    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzzhc\" (UniqueName: \"kubernetes.io/projected/0599c4e4-3db5-42b2-bd74-2c874ab74383-kube-api-access-rzzhc\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:16:31.522055 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.522049    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22q58\" (UniqueName: \"kubernetes.io/projected/e0c7e7cd-abcd-429b-b556-947ac5d9402a-kube-api-access-22q58\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.560367 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560338    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-cni-netd\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560430 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560373    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-run-ovn-kubernetes\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560430 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560390    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-registration-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.560430 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560406    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-socket-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.560430 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560426    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-log-socket\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560550 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560431    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-cni-netd\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560550 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560434    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-run-ovn-kubernetes\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560550 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560452    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-etc-selinux\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.560550 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560467    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-registration-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.560550 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560490    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-cni-bin\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560550 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560516    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-slash\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560550 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560522    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-log-socket\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560550 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560536    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-etc-selinux\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560556    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-socket-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560539    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39640be1-9711-4cef-a9a2-430ea40c7c18-ovn-node-metrics-cert\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560579    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-cni-bin\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560611    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-ovnkube-config\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560646    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560570    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-slash\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560661    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-env-overrides\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560676    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-ovnkube-script-lib\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560698    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560710    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560727    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3aef6eaf-2dde-4706-bcf9-77bf9c447028-agent-certs\") pod \"konnectivity-agent-6x62l\" (UID: \"3aef6eaf-2dde-4706-bcf9-77bf9c447028\") " pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560756    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-device-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560821    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-kubelet\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.560871 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560811    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560888    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-kubelet\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560887    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-sys-fs\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560941    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-device-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560953    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-run-netns\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560977    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-systemd\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.560987    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-sys-fs\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561002    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-etc-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561019    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-systemd\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561021    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-run-netns\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561026    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-ovn\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561047    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-etc-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561101    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-ovn\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561114    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsm6h\" (UniqueName: \"kubernetes.io/projected/39640be1-9711-4cef-a9a2-430ea40c7c18-kube-api-access-tsm6h\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561142    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl5th\" (UniqueName: \"kubernetes.io/projected/e1685aa8-41b4-4899-ad2d-6e03e2969d72-kube-api-access-bl5th\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561137    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.561515 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561169    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-systemd-units\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.562125 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561155    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-env-overrides\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.562125 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561195    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-var-lib-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.562125 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561197    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-ovnkube-config\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.562125 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561225    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-systemd-units\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.562125 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561227    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3aef6eaf-2dde-4706-bcf9-77bf9c447028-konnectivity-ca\") pod \"konnectivity-agent-6x62l\" (UID: \"3aef6eaf-2dde-4706-bcf9-77bf9c447028\") " pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:16:31.562125 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561254    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-var-lib-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.562125 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561267    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-node-log\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.562125 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561319    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-node-log\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.562125 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561613    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3aef6eaf-2dde-4706-bcf9-77bf9c447028-konnectivity-ca\") pod \"konnectivity-agent-6x62l\" (UID: \"3aef6eaf-2dde-4706-bcf9-77bf9c447028\") " pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:16:31.562125 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.561790    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-ovnkube-script-lib\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.562393 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.562364    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39640be1-9711-4cef-a9a2-430ea40c7c18-ovn-node-metrics-cert\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.562765 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.562747    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3aef6eaf-2dde-4706-bcf9-77bf9c447028-agent-certs\") pod \"konnectivity-agent-6x62l\" (UID: \"3aef6eaf-2dde-4706-bcf9-77bf9c447028\") " pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:16:31.624163 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.624133    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsm6h\" (UniqueName: \"kubernetes.io/projected/39640be1-9711-4cef-a9a2-430ea40c7c18-kube-api-access-tsm6h\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.624298 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.624284    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl5th\" (UniqueName: \"kubernetes.io/projected/e1685aa8-41b4-4899-ad2d-6e03e2969d72-kube-api-access-bl5th\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.641886 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.641867    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:16:31.646310 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.646253    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:16:31.653303 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:31.653280    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f36ad6_52cd_4239_94eb_22b8f3de2a26.slice/crio-226a4227a8a92d34d1406056e151c3db79dbb29244396dd744f6829da5e23522 WatchSource:0}: Error finding container 226a4227a8a92d34d1406056e151c3db79dbb29244396dd744f6829da5e23522: Status 404 returned error can't find the container with id 226a4227a8a92d34d1406056e151c3db79dbb29244396dd744f6829da5e23522
Nov 04 03:16:31.655102 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:31.655075    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f6fbb1_af6e_43e4_a671_68a6c3691463.slice/crio-cee6bfcaf1bd44ae12c618ae6c1e053472d1563db85a7d7396a7ecd8d92e5737 WatchSource:0}: Error finding container cee6bfcaf1bd44ae12c618ae6c1e053472d1563db85a7d7396a7ecd8d92e5737: Status 404 returned error can't find the container with id cee6bfcaf1bd44ae12c618ae6c1e053472d1563db85a7d7396a7ecd8d92e5737
Nov 04 03:16:31.665043 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.665023    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qsm2l"
Nov 04 03:16:31.666097 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.666081    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:16:31.668197 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.668177    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:16:31.671777 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.671758    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:16:31.673866 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:31.673830    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0c7e7cd_abcd_429b_b556_947ac5d9402a.slice/crio-824d2cbbc83cbdfeeb4918c559ec65a0d7f6d1274543f433770e0ab40fd6126c WatchSource:0}: Error finding container 824d2cbbc83cbdfeeb4918c559ec65a0d7f6d1274543f433770e0ab40fd6126c: Status 404 returned error can't find the container with id 824d2cbbc83cbdfeeb4918c559ec65a0d7f6d1274543f433770e0ab40fd6126c
Nov 04 03:16:31.675357 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.675329    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:16:31.677116 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:31.677090    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7586e6a7_b118_4cef_9c67_56e5a5d4eb04.slice/crio-d8e001ce1904ee2e790a3fcec8c9019c4cf9adec7600e7653ba572e5946be7d8 WatchSource:0}: Error finding container d8e001ce1904ee2e790a3fcec8c9019c4cf9adec7600e7653ba572e5946be7d8: Status 404 returned error can't find the container with id d8e001ce1904ee2e790a3fcec8c9019c4cf9adec7600e7653ba572e5946be7d8
Nov 04 03:16:31.680459 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:31.680441    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4f10f3e_8fd4_4e4e_acc1_3c5622796acd.slice/crio-649cc2ad8030b26a82eee98ded3ee17b4e15056a72a7fd0753a3945628043328 WatchSource:0}: Error finding container 649cc2ad8030b26a82eee98ded3ee17b4e15056a72a7fd0753a3945628043328: Status 404 returned error can't find the container with id 649cc2ad8030b26a82eee98ded3ee17b4e15056a72a7fd0753a3945628043328
Nov 04 03:16:31.684803 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:31.684786    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0599c4e4_3db5_42b2_bd74_2c874ab74383.slice/crio-ef921ad20b1e70cfaac7f27cd34d3b0667f758fa4e4241d6be625e40cd6dc350 WatchSource:0}: Error finding container ef921ad20b1e70cfaac7f27cd34d3b0667f758fa4e4241d6be625e40cd6dc350: Status 404 returned error can't find the container with id ef921ad20b1e70cfaac7f27cd34d3b0667f758fa4e4241d6be625e40cd6dc350
Nov 04 03:16:31.685301 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.685287    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:16:31.685534 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:31.685517    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aef6eaf_2dde_4706_bcf9_77bf9c447028.slice/crio-d67c059c089997458d92b6729f9515c79085d26cd13de7bcc543b3ba17a4497e WatchSource:0}: Error finding container d67c059c089997458d92b6729f9515c79085d26cd13de7bcc543b3ba17a4497e: Status 404 returned error can't find the container with id d67c059c089997458d92b6729f9515c79085d26cd13de7bcc543b3ba17a4497e
Nov 04 03:16:31.693552 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:31.693534    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1685aa8_41b4_4899_ad2d_6e03e2969d72.slice/crio-62a22db6e28f2b130c91e53743021384e0ebcf95cfb58541d96f8a57750c4b2d WatchSource:0}: Error finding container 62a22db6e28f2b130c91e53743021384e0ebcf95cfb58541d96f8a57750c4b2d: Status 404 returned error can't find the container with id 62a22db6e28f2b130c91e53743021384e0ebcf95cfb58541d96f8a57750c4b2d
Nov 04 03:16:31.701621 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.701606    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:31.709857 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:31.709816    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39640be1_9711_4cef_a9a2_430ea40c7c18.slice/crio-c4429e7010d037d1175d80fa1f248377ab4d6a97caf4ef8888ab956eb40d27bf WatchSource:0}: Error finding container c4429e7010d037d1175d80fa1f248377ab4d6a97caf4ef8888ab956eb40d27bf: Status 404 returned error can't find the container with id c4429e7010d037d1175d80fa1f248377ab4d6a97caf4ef8888ab956eb40d27bf
Nov 04 03:16:31.964536 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.964455    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:31.964536 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:31.964513    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:31.964711 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.964589    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:31.964711 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.964605    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:31.964711 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.964647    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs podName:ae2e6da6-1e6d-4b54-8136-359490273f9c nodeName:}" failed. No retries permitted until 2026-11-04 03:16:32.964632423 +0000 UTC m=+4.146130662 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs") pod "network-metrics-daemon-rz4mh" (UID: "ae2e6da6-1e6d-4b54-8136-359490273f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:31.964711 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:31.964661    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret podName:39429a51-69a0-4bce-9029-773d5aad2a87 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:32.964655178 +0000 UTC m=+4.146153404 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret") pod "global-pull-secret-syncer-tmrtb" (UID: "39429a51-69a0-4bce-9029-773d5aad2a87") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:32.065312 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.065286    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lswx\" (UniqueName: \"kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx\") pod \"network-check-target-5zxn5\" (UID: \"7566445c-b5ac-424d-aa38-a2c5b2e0de21\") " pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:32.065439 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:32.065391    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:32.065439 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:32.065403    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:32.065439 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:32.065411    2574 projected.go:196] Error preparing data for projected volume kube-api-access-5lswx for pod openshift-network-diagnostics/network-check-target-5zxn5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:32.065550 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:32.065452    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx podName:7566445c-b5ac-424d-aa38-a2c5b2e0de21 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:33.065439823 +0000 UTC m=+4.246938049 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5lswx" (UniqueName: "kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx") pod "network-check-target-5zxn5" (UID: "7566445c-b5ac-424d-aa38-a2c5b2e0de21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:32.375449 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.375418    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l74dq" event={"ID":"7586e6a7-b118-4cef-9c67-56e5a5d4eb04","Type":"ContainerStarted","Data":"d8e001ce1904ee2e790a3fcec8c9019c4cf9adec7600e7653ba572e5946be7d8"}
Nov 04 03:16:32.376628 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.376601    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vg574" event={"ID":"39640be1-9711-4cef-a9a2-430ea40c7c18","Type":"ContainerStarted","Data":"c4429e7010d037d1175d80fa1f248377ab4d6a97caf4ef8888ab956eb40d27bf"}
Nov 04 03:16:32.377650 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.377629    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px" event={"ID":"e1685aa8-41b4-4899-ad2d-6e03e2969d72","Type":"ContainerStarted","Data":"62a22db6e28f2b130c91e53743021384e0ebcf95cfb58541d96f8a57750c4b2d"}
Nov 04 03:16:32.378928 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.378901    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8r4z6" event={"ID":"0599c4e4-3db5-42b2-bd74-2c874ab74383","Type":"ContainerStarted","Data":"ef921ad20b1e70cfaac7f27cd34d3b0667f758fa4e4241d6be625e40cd6dc350"}
Nov 04 03:16:32.380152 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.380129    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2tvj" event={"ID":"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd","Type":"ContainerStarted","Data":"649cc2ad8030b26a82eee98ded3ee17b4e15056a72a7fd0753a3945628043328"}
Nov 04 03:16:32.381310 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.381285    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qsm2l" event={"ID":"e0c7e7cd-abcd-429b-b556-947ac5d9402a","Type":"ContainerStarted","Data":"824d2cbbc83cbdfeeb4918c559ec65a0d7f6d1274543f433770e0ab40fd6126c"}
Nov 04 03:16:32.382449 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.382429    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gccvl" event={"ID":"e1f6fbb1-af6e-43e4-a671-68a6c3691463","Type":"ContainerStarted","Data":"cee6bfcaf1bd44ae12c618ae6c1e053472d1563db85a7d7396a7ecd8d92e5737"}
Nov 04 03:16:32.383616 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.383595    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n6s5j" event={"ID":"50f36ad6-52cd-4239-94eb-22b8f3de2a26","Type":"ContainerStarted","Data":"226a4227a8a92d34d1406056e151c3db79dbb29244396dd744f6829da5e23522"}
Nov 04 03:16:32.384802 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.384779    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6x62l" event={"ID":"3aef6eaf-2dde-4706-bcf9-77bf9c447028","Type":"ContainerStarted","Data":"d67c059c089997458d92b6729f9515c79085d26cd13de7bcc543b3ba17a4497e"}
Nov 04 03:16:32.439604 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.439565    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:11:31 +0000 UTC" deadline="2028-04-19 14:36:54.515871278 +0000 UTC"
Nov 04 03:16:32.439604 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.439608    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12779h20m22.076265736s"
Nov 04 03:16:32.972448 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.972415    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:32.972625 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:32.972481    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:32.972694 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:32.972627    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:32.972694 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:32.972686    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs podName:ae2e6da6-1e6d-4b54-8136-359490273f9c nodeName:}" failed. No retries permitted until 2026-11-04 03:16:34.972667825 +0000 UTC m=+6.154166065 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs") pod "network-metrics-daemon-rz4mh" (UID: "ae2e6da6-1e6d-4b54-8136-359490273f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:32.972805 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:32.972760    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:32.972805 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:32.972794    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret podName:39429a51-69a0-4bce-9029-773d5aad2a87 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:34.972782999 +0000 UTC m=+6.154281226 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret") pod "global-pull-secret-syncer-tmrtb" (UID: "39429a51-69a0-4bce-9029-773d5aad2a87") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:33.073653 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:33.073624    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lswx\" (UniqueName: \"kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx\") pod \"network-check-target-5zxn5\" (UID: \"7566445c-b5ac-424d-aa38-a2c5b2e0de21\") " pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:33.073874 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:33.073854    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:33.073939 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:33.073883    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:33.073939 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:33.073896    2574 projected.go:196] Error preparing data for projected volume kube-api-access-5lswx for pod openshift-network-diagnostics/network-check-target-5zxn5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:33.074045 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:33.073951    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx podName:7566445c-b5ac-424d-aa38-a2c5b2e0de21 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:35.073934879 +0000 UTC m=+6.255433119 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5lswx" (UniqueName: "kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx") pod "network-check-target-5zxn5" (UID: "7566445c-b5ac-424d-aa38-a2c5b2e0de21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:33.368001 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:33.367960    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:33.368184 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:33.368146    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:33.368250 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:33.368243    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:33.368332 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:33.368314    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:33.368387 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:33.368375    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:33.368455 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:33.368439    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:33.440380 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:33.440317    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:11:31 +0000 UTC" deadline="2028-06-21 08:13:37.274921535 +0000 UTC"
Nov 04 03:16:33.440380 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:33.440379    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14284h57m3.834547086s"
Nov 04 03:16:34.881632 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:16:34.881596    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ddb4465d8ebfdcb6c7be795263771f.slice/crio-6fb613bc7c958935a8d23066b5bc51099623fab47f72c4dd27825c6579aba8b8 WatchSource:0}: Error finding container 6fb613bc7c958935a8d23066b5bc51099623fab47f72c4dd27825c6579aba8b8: Status 404 returned error can't find the container with id 6fb613bc7c958935a8d23066b5bc51099623fab47f72c4dd27825c6579aba8b8
Nov 04 03:16:34.988760 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:34.988724    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:34.988914 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:34.988793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:34.988976 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:34.988932    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:34.989027 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:34.988990    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs podName:ae2e6da6-1e6d-4b54-8136-359490273f9c nodeName:}" failed. No retries permitted until 2026-11-04 03:16:38.988970227 +0000 UTC m=+10.170468673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs") pod "network-metrics-daemon-rz4mh" (UID: "ae2e6da6-1e6d-4b54-8136-359490273f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:34.989090 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:34.989058    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:34.989142 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:34.989095    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret podName:39429a51-69a0-4bce-9029-773d5aad2a87 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:38.989083032 +0000 UTC m=+10.170581274 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret") pod "global-pull-secret-syncer-tmrtb" (UID: "39429a51-69a0-4bce-9029-773d5aad2a87") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:35.089107 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:35.089078    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lswx\" (UniqueName: \"kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx\") pod \"network-check-target-5zxn5\" (UID: \"7566445c-b5ac-424d-aa38-a2c5b2e0de21\") " pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:35.089281 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:35.089258    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:35.089337 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:35.089295    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:35.089337 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:35.089310    2574 projected.go:196] Error preparing data for projected volume kube-api-access-5lswx for pod openshift-network-diagnostics/network-check-target-5zxn5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:35.089464 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:35.089449    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx podName:7566445c-b5ac-424d-aa38-a2c5b2e0de21 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:39.089396605 +0000 UTC m=+10.270894834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5lswx" (UniqueName: "kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx") pod "network-check-target-5zxn5" (UID: "7566445c-b5ac-424d-aa38-a2c5b2e0de21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:35.372740 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:35.369589    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:35.372740 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:35.369772    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:35.372740 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:35.369952    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:35.372740 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:35.370043    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:35.372740 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:35.370094    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:35.372740 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:35.370162    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:35.412124 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:35.412014    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal" event={"ID":"72ddb4465d8ebfdcb6c7be795263771f","Type":"ContainerStarted","Data":"6fb613bc7c958935a8d23066b5bc51099623fab47f72c4dd27825c6579aba8b8"}
Nov 04 03:16:35.417768 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:35.417738    2574 generic.go:356] "Generic (PLEG): container finished" podID="4d7792c1def8a40547a57207fd144414" containerID="a5112290b882e1bb49f444afa2a5ef8a39966795db3695790e961d246561b9a2" exitCode=0
Nov 04 03:16:35.417919 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:35.417827    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal" event={"ID":"4d7792c1def8a40547a57207fd144414","Type":"ContainerDied","Data":"a5112290b882e1bb49f444afa2a5ef8a39966795db3695790e961d246561b9a2"}
Nov 04 03:16:35.424649 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:35.424594    2574 generic.go:356] "Generic (PLEG): container finished" podID="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" containerID="d6221af860fc44d21107155b2c0f6b3b5bc75cfe9ae5cbddb9bd475f87b145cb" exitCode=0
Nov 04 03:16:35.424649 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:35.424632    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2tvj" event={"ID":"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd","Type":"ContainerDied","Data":"d6221af860fc44d21107155b2c0f6b3b5bc75cfe9ae5cbddb9bd475f87b145cb"}
Nov 04 03:16:36.433692 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:36.433648    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal" event={"ID":"4d7792c1def8a40547a57207fd144414","Type":"ContainerStarted","Data":"ac555a88ec118d69ff1a06ad35df562db17bd05e6c25f8d9ff71ce49269c5e7d"}
Nov 04 03:16:36.439181 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:36.439149    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6x62l" event={"ID":"3aef6eaf-2dde-4706-bcf9-77bf9c447028","Type":"ContainerStarted","Data":"21159307c06f62413af7f5da3d995a807f64792c76b280e1fadbf91627bdd2fd"}
Nov 04 03:16:36.673002 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:36.672922    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal" podStartSLOduration=6.672900553 podStartE2EDuration="6.672900553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:36.553548817 +0000 UTC m=+7.735047066" watchObservedRunningTime="2026-11-04 03:16:36.672900553 +0000 UTC m=+7.854398803"
Nov 04 03:16:37.367902 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:37.367364    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:37.367902 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:37.367467    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:37.367902 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:37.367759    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:37.367902 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:37.367837    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:37.368331 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:37.367952    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:37.368331 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:37.368024    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:38.426162 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:38.426121    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:16:38.447095 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:38.446899    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px" event={"ID":"e1685aa8-41b4-4899-ad2d-6e03e2969d72","Type":"ContainerStarted","Data":"066abaa1b4442b83bca4dadbc2ab13364d3388452f68cc3b43f1aa434293a598"}
Nov 04 03:16:38.666706 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:38.666625    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:16:38.667145 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:38.667123    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:16:38.786867 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:38.786793    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6x62l" podStartSLOduration=5.520440893 podStartE2EDuration="9.786770672s" totalImagesPullingTime="4.266329779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:36.673973532 +0000 UTC m=+7.855471780" watchObservedRunningTime="2026-11-04 03:16:38.786770672 +0000 UTC m=+9.968268921"
Nov 04 03:16:39.022240 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:39.022206    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:39.022439 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:39.022274    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:39.022439 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:39.022425    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:39.022549 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:39.022479    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs podName:ae2e6da6-1e6d-4b54-8136-359490273f9c nodeName:}" failed. No retries permitted until 2026-11-04 03:16:47.02246629 +0000 UTC m=+18.203964516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs") pod "network-metrics-daemon-rz4mh" (UID: "ae2e6da6-1e6d-4b54-8136-359490273f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:39.022866 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:39.022830    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:39.022969 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:39.022898    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret podName:39429a51-69a0-4bce-9029-773d5aad2a87 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:47.02288413 +0000 UTC m=+18.204382357 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret") pod "global-pull-secret-syncer-tmrtb" (UID: "39429a51-69a0-4bce-9029-773d5aad2a87") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:39.124017 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:39.123986    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lswx\" (UniqueName: \"kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx\") pod \"network-check-target-5zxn5\" (UID: \"7566445c-b5ac-424d-aa38-a2c5b2e0de21\") " pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:39.124162 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:39.124132    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:39.124162 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:39.124148    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:39.124162 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:39.124160    2574 projected.go:196] Error preparing data for projected volume kube-api-access-5lswx for pod openshift-network-diagnostics/network-check-target-5zxn5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:39.124318 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:39.124204    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx podName:7566445c-b5ac-424d-aa38-a2c5b2e0de21 nodeName:}" failed. No retries permitted until 2026-11-04 03:16:47.124192057 +0000 UTC m=+18.305690287 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5lswx" (UniqueName: "kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx") pod "network-check-target-5zxn5" (UID: "7566445c-b5ac-424d-aa38-a2c5b2e0de21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:39.379337 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:39.379262    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:39.379485 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:39.379429    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:39.379926 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:39.379904    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:39.380062 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:39.380030    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:39.380159 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:39.380140    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:39.380302 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:39.380282    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:39.461924 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:39.458035    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8r4z6" event={"ID":"0599c4e4-3db5-42b2-bd74-2c874ab74383","Type":"ContainerStarted","Data":"93a5ff6ad779f429553e54095e39d479e415391d2cbe7c96a10393ae9d925583"}
Nov 04 03:16:39.461924 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:39.458514    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:16:39.578252 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:39.577640    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8r4z6" podStartSLOduration=3.958403147 podStartE2EDuration="10.57762292s" totalImagesPullingTime="6.619219773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:39.576783574 +0000 UTC m=+10.758281822" watchObservedRunningTime="2026-11-04 03:16:39.57762292 +0000 UTC m=+10.759121169"
Nov 04 03:16:40.246985 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:40.246960    2574 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 03:16:40.454794 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:40.454670    2574 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T03:16:40.246978094Z","UUID":"c80f2486-9d12-45b9-9b1f-b9d933637cc6","Handler":null,"Name":"","Endpoint":""}
Nov 04 03:16:40.456545 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:40.456439    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 03:16:40.456661 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:40.456556    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 03:16:40.467291 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:40.467257    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal" event={"ID":"72ddb4465d8ebfdcb6c7be795263771f","Type":"ContainerStarted","Data":"837fb68afd690ffdac3a41605531db338b6a09ed98d2eb67467be85386374a18"}
Nov 04 03:16:40.472097 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:40.472064    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px" event={"ID":"e1685aa8-41b4-4899-ad2d-6e03e2969d72","Type":"ContainerStarted","Data":"2e084f3b08b9cc7d35e4e1c1d52a079d8ed6aad3085fbfb6a7a39d67d7812516"}
Nov 04 03:16:40.475210 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:40.475184    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qsm2l" event={"ID":"e0c7e7cd-abcd-429b-b556-947ac5d9402a","Type":"ContainerStarted","Data":"3983416df603bae92d6c1c67805e9860fb95ae5c75796558cfce23598c4c35b1"}
Nov 04 03:16:40.708360 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:40.708308    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal" podStartSLOduration=10.708289401 podStartE2EDuration="10.708289401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:40.586243411 +0000 UTC m=+11.767741662" watchObservedRunningTime="2026-11-04 03:16:40.708289401 +0000 UTC m=+11.889787651"
Nov 04 03:16:40.708855 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:40.708817    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-qsm2l" podStartSLOduration=3.055812499 podStartE2EDuration="11.708811051s" totalImagesPullingTime="8.652998552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:40.708137984 +0000 UTC m=+11.889636232" watchObservedRunningTime="2026-11-04 03:16:40.708811051 +0000 UTC m=+11.890309299"
Nov 04 03:16:41.368132 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:41.368096    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:41.368279 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:41.368227    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:41.368607 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:41.368593    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:41.368673 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:41.368658    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:41.368785 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:41.368775    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:41.368875 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:41.368860    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:41.478093 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:41.478059    2574 generic.go:356] "Generic (PLEG): container finished" podID="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" containerID="126f6eb7d0a6858661b9fffbecef6e49b12c4281d35a25323d4b625225dbac32" exitCode=0
Nov 04 03:16:41.478522 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:41.478146    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2tvj" event={"ID":"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd","Type":"ContainerDied","Data":"126f6eb7d0a6858661b9fffbecef6e49b12c4281d35a25323d4b625225dbac32"}
Nov 04 03:16:41.479743 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:41.479713    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gccvl" event={"ID":"e1f6fbb1-af6e-43e4-a671-68a6c3691463","Type":"ContainerStarted","Data":"0c92fe5a3c7f83411493dda0849b0fa2ea1cdfd15bc127689bb695e17d4b0062"}
Nov 04 03:16:41.725322 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:41.725224    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-gccvl" podStartSLOduration=3.887922704 podStartE2EDuration="12.725205538s" totalImagesPullingTime="8.837282834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:41.724651224 +0000 UTC m=+12.906149473" watchObservedRunningTime="2026-11-04 03:16:41.725205538 +0000 UTC m=+12.906703788"
Nov 04 03:16:42.484571 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:42.484537    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px" event={"ID":"e1685aa8-41b4-4899-ad2d-6e03e2969d72","Type":"ContainerStarted","Data":"f2060e159bf7d218c961aab2e660fd0871219a129733394574e43eea976d1430"}
Nov 04 03:16:42.486312 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:42.486277    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n6s5j" event={"ID":"50f36ad6-52cd-4239-94eb-22b8f3de2a26","Type":"ContainerStarted","Data":"026a8e115befd747728f881d841e0f202a82d35764d4f1cb713dcc47e836743b"}
Nov 04 03:16:42.607433 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:42.607127    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px" podStartSLOduration=3.7615614859999997 podStartE2EDuration="13.607112811s" totalImagesPullingTime="9.845551325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:42.606716721 +0000 UTC m=+13.788214971" watchObservedRunningTime="2026-11-04 03:16:42.607112811 +0000 UTC m=+13.788611063"
Nov 04 03:16:42.725700 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:42.725642    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-n6s5j" podStartSLOduration=4.714513544 podStartE2EDuration="13.725624062s" totalImagesPullingTime="9.011110518s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:42.725229817 +0000 UTC m=+13.906728064" watchObservedRunningTime="2026-11-04 03:16:42.725624062 +0000 UTC m=+13.907122310"
Nov 04 03:16:43.367324 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:43.367296    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:43.370315 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:43.367357    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:43.370315 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:43.367755    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:43.370315 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:43.367799    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:43.370315 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:43.367928    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:43.370582 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:43.370479    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:43.490630 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:43.490595    2574 generic.go:356] "Generic (PLEG): container finished" podID="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" containerID="d93f00c373a9e1a7a49332d5b0c3ef8dd9ecd97e69b160fdd56a013d07ccd8d9" exitCode=0
Nov 04 03:16:43.491222 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:43.490682    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2tvj" event={"ID":"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd","Type":"ContainerDied","Data":"d93f00c373a9e1a7a49332d5b0c3ef8dd9ecd97e69b160fdd56a013d07ccd8d9"}
Nov 04 03:16:43.492290 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:43.492267    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l74dq" event={"ID":"7586e6a7-b118-4cef-9c67-56e5a5d4eb04","Type":"ContainerStarted","Data":"cdbe5486c4280d077a714afc48090cf82e06be526fb9841cbf60452b369171a0"}
Nov 04 03:16:43.739414 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:43.739367    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l74dq" podStartSLOduration=3.599637358 podStartE2EDuration="14.739353941s" totalImagesPullingTime="11.139716583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:43.738925767 +0000 UTC m=+14.920424005" watchObservedRunningTime="2026-11-04 03:16:43.739353941 +0000 UTC m=+14.920852189"
Nov 04 03:16:45.367996 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:45.367968    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:45.368646 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:45.368007    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:45.368646 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:45.367966    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:45.368646 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:45.368093    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:45.368646 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:45.368171    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:45.368646 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:45.368216    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:45.500225 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:45.499492    2574 generic.go:356] "Generic (PLEG): container finished" podID="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" containerID="860591cc6810b7f28e163829b6db691932a35e0698eed0699c30822dff3960d1" exitCode=0
Nov 04 03:16:45.500225 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:45.499540    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2tvj" event={"ID":"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd","Type":"ContainerDied","Data":"860591cc6810b7f28e163829b6db691932a35e0698eed0699c30822dff3960d1"}
Nov 04 03:16:47.083706 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:47.083491    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:47.084193 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:47.083748    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:47.084193 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:47.083621    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:47.084193 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:47.083839    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret podName:39429a51-69a0-4bce-9029-773d5aad2a87 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:03.083818932 +0000 UTC m=+34.265317177 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret") pod "global-pull-secret-syncer-tmrtb" (UID: "39429a51-69a0-4bce-9029-773d5aad2a87") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:16:47.084193 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:47.083893    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:47.084193 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:47.083930    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs podName:ae2e6da6-1e6d-4b54-8136-359490273f9c nodeName:}" failed. No retries permitted until 2026-11-04 03:17:03.083918365 +0000 UTC m=+34.265416603 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs") pod "network-metrics-daemon-rz4mh" (UID: "ae2e6da6-1e6d-4b54-8136-359490273f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:16:47.184102 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:47.184066    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lswx\" (UniqueName: \"kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx\") pod \"network-check-target-5zxn5\" (UID: \"7566445c-b5ac-424d-aa38-a2c5b2e0de21\") " pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:47.184277 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:47.184227    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:16:47.184277 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:47.184252    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:16:47.184277 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:47.184265    2574 projected.go:196] Error preparing data for projected volume kube-api-access-5lswx for pod openshift-network-diagnostics/network-check-target-5zxn5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:47.184509 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:47.184326    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx podName:7566445c-b5ac-424d-aa38-a2c5b2e0de21 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:03.184308 +0000 UTC m=+34.365806244 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5lswx" (UniqueName: "kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx") pod "network-check-target-5zxn5" (UID: "7566445c-b5ac-424d-aa38-a2c5b2e0de21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:16:47.367277 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:47.367191    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:47.367396 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:47.367309    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:47.367698 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:47.367681    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:47.367780 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:47.367762    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:47.367922 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:47.367904    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:47.368022 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:47.368004    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:48.508121 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:48.507946    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vg574" event={"ID":"39640be1-9711-4cef-a9a2-430ea40c7c18","Type":"ContainerStarted","Data":"478e718b5dee81c0f6d78c40e0d5b7956ba581891846645bc53072415b781a7c"}
Nov 04 03:16:48.508554 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:48.508170    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vg574" event={"ID":"39640be1-9711-4cef-a9a2-430ea40c7c18","Type":"ContainerStarted","Data":"5367bed6349ebdabacfd702e8cda505932e82cbfbcd9c082cc16676dce2e4616"}
Nov 04 03:16:48.508554 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:48.508212    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vg574" event={"ID":"39640be1-9711-4cef-a9a2-430ea40c7c18","Type":"ContainerStarted","Data":"75c07092e0897c5e09c9e90c4085898ccfb96839f9d7a6bb4ffdb66d2433578a"}
Nov 04 03:16:48.508554 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:48.508252    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vg574" event={"ID":"39640be1-9711-4cef-a9a2-430ea40c7c18","Type":"ContainerStarted","Data":"bcac687c63a2be0a4f229c91a53ae1af42835d1688981e337cb00e21dd6a6e17"}
Nov 04 03:16:48.508554 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:48.508296    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vg574" event={"ID":"39640be1-9711-4cef-a9a2-430ea40c7c18","Type":"ContainerStarted","Data":"6c5d30f6323378f30cd635a7ed30cb9304fe94172b39b47c02bb982bc00f5ac4"}
Nov 04 03:16:48.508554 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:48.508342    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vg574" event={"ID":"39640be1-9711-4cef-a9a2-430ea40c7c18","Type":"ContainerStarted","Data":"12a20b07338842c0f990f5deb84f064d65680a16869c9758e6b3578f18c7c4d2"}
Nov 04 03:16:48.510752 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:48.510723    2574 generic.go:356] "Generic (PLEG): container finished" podID="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" containerID="82b2d1cf56f50ffbe09754622d8198251be6e521c938c5fc251469be1c039441" exitCode=0
Nov 04 03:16:48.510881 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:48.510761    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2tvj" event={"ID":"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd","Type":"ContainerDied","Data":"82b2d1cf56f50ffbe09754622d8198251be6e521c938c5fc251469be1c039441"}
Nov 04 03:16:49.368270 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:49.368241    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:49.368411 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:49.368319    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:49.368480 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:49.368424    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:49.368548 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:49.368537    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:49.368580 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:49.368548    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:49.368616 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:49.368599    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:49.514886 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:49.514853    2574 generic.go:356] "Generic (PLEG): container finished" podID="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" containerID="621ded60eed8678e8b1f9a99fa5f2bbe6667a8789b69b90d5e14fe585068b5e5" exitCode=0
Nov 04 03:16:49.515219 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:49.514904    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2tvj" event={"ID":"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd","Type":"ContainerDied","Data":"621ded60eed8678e8b1f9a99fa5f2bbe6667a8789b69b90d5e14fe585068b5e5"}
Nov 04 03:16:50.520306 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:50.520267    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vg574" event={"ID":"39640be1-9711-4cef-a9a2-430ea40c7c18","Type":"ContainerStarted","Data":"40b271e775aa0cd9cac78fefdd0b042e8fd3ef637526c10904e9037d958588f6"}
Nov 04 03:16:50.523373 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:50.523343    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2tvj" event={"ID":"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd","Type":"ContainerStarted","Data":"5539469a6766013d982d247378b89c3ed1a69bad1c48eae2c2b33d0df0f549ae"}
Nov 04 03:16:50.650947 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:50.650839    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s2tvj" podStartSLOduration=7.17981756 podStartE2EDuration="21.650824736s" totalImagesPullingTime="14.471007176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:50.650233666 +0000 UTC m=+21.831731913" watchObservedRunningTime="2026-11-04 03:16:50.650824736 +0000 UTC m=+21.832322986"
Nov 04 03:16:51.367549 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:51.367514    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:51.367549 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:51.367534    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:51.367787 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:51.367618    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:51.367787 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:51.367632    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:51.367787 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:51.367731    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:51.367951 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:51.367815    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:52.531096 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:52.531058    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vg574" event={"ID":"39640be1-9711-4cef-a9a2-430ea40c7c18","Type":"ContainerStarted","Data":"9d976af29f93fd3a3771c98b97c0e85b432e60e01d2e21fad8dea6f60cb3cb15"}
Nov 04 03:16:52.532031 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:52.531347    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:52.546145 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:52.546120    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:52.820325 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:52.820278    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vg574" podStartSLOduration=7.598010498 podStartE2EDuration="23.820265658s" totalImagesPullingTime="16.22225516s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:16:52.796977522 +0000 UTC m=+23.978475771" watchObservedRunningTime="2026-11-04 03:16:52.820265658 +0000 UTC m=+24.001763906"
Nov 04 03:16:53.367077 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:53.367044    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:53.367077 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:53.367057    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:53.367311 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:53.367144    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:53.367311 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:53.367160    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:53.367311 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:53.367214    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:53.367311 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:53.367277    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:53.533239 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:53.533212    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:16:53.533750 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:53.533590    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:53.548435 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:53.548414    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:16:54.535359 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:54.535330    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:16:55.093794 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:55.093766    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tmrtb"]
Nov 04 03:16:55.093961 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:55.093868    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:55.093961 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:55.093946    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:55.096784 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:55.096759    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5zxn5"]
Nov 04 03:16:55.096928 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:55.096873    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:55.096987 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:55.096965    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:55.145964 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:55.145936    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rz4mh"]
Nov 04 03:16:55.146053 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:55.146022    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:55.146110 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:55.146094    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:55.537124 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:55.537094    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:16:56.367517 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:56.367482    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:56.367658 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:56.367592    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:56.367658 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:56.367642    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:56.367758 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:56.367741    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:57.367882 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:57.367796    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:57.368245 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:57.367913    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:16:58.367353 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:58.367320    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:16:58.367498 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:58.367432    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:16:58.367562 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:58.367490    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:16:58.367632 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:58.367607    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:16:59.368789 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:16:59.368760    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:16:59.369198 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:16:59.368861    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tmrtb" podUID="39429a51-69a0-4bce-9029-773d5aad2a87"
Nov 04 03:17:00.367772 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:00.367733    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:00.367988 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:00.367887    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rz4mh" podUID="ae2e6da6-1e6d-4b54-8136-359490273f9c"
Nov 04 03:17:00.367988 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:00.367927    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:17:00.368105 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:00.368015    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zxn5" podUID="7566445c-b5ac-424d-aa38-a2c5b2e0de21"
Nov 04 03:17:01.058525 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.058487    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:01.116221 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.116197    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeReady"
Nov 04 03:17:01.116332 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.116303    2574 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 03:17:01.237002 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.236936    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2kdk2"]
Nov 04 03:17:01.266506 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.266478    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:01.282361 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.282337    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ckjtr"]
Nov 04 03:17:01.282492 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.282471    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2kdk2"
Nov 04 03:17:01.299758 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.299739    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2kdk2"]
Nov 04 03:17:01.299857 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.299822    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:01.299901 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.299874    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:01.299960 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.299946    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ckjtr"]
Nov 04 03:17:01.306535 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.306518    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:17:01.308664 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.308646    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:01.337568 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.337543    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 03:17:01.337642 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.337594    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 03:17:01.337642 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.337622    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-mhdb8\""
Nov 04 03:17:01.337937 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.337923    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 03:17:01.354614 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.354600    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 03:17:01.354660 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.354635    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 03:17:01.354660 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.354648    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vjndb\""
Nov 04 03:17:01.367854 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.367825    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:17:01.394125 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.394102    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36af1207-9951-4160-bb36-6dd100a05abb-tmp-dir\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:01.394221 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.394129    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjqlf\" (UniqueName: \"kubernetes.io/projected/6cec1f32-6464-4117-b098-2ed5b3c9cf64-kube-api-access-vjqlf\") pod \"ingress-canary-2kdk2\" (UID: \"6cec1f32-6464-4117-b098-2ed5b3c9cf64\") " pod="openshift-ingress-canary/ingress-canary-2kdk2"
Nov 04 03:17:01.394221 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.394166    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/36af1207-9951-4160-bb36-6dd100a05abb-config-volume\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:01.394396 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.394377    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:01.394450 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.394412    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcpp8\" (UniqueName: \"kubernetes.io/projected/36af1207-9951-4160-bb36-6dd100a05abb-kube-api-access-kcpp8\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:01.394503 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.394449    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cec1f32-6464-4117-b098-2ed5b3c9cf64-cert\") pod \"ingress-canary-2kdk2\" (UID: \"6cec1f32-6464-4117-b098-2ed5b3c9cf64\") " pod="openshift-ingress-canary/ingress-canary-2kdk2"
Nov 04 03:17:01.438695 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.438675    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 03:17:01.438804 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.438707    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zsv6g\""
Nov 04 03:17:01.468738 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.468716    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:01.494733 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.494709    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36af1207-9951-4160-bb36-6dd100a05abb-tmp-dir\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:01.494893 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.494742    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjqlf\" (UniqueName: \"kubernetes.io/projected/6cec1f32-6464-4117-b098-2ed5b3c9cf64-kube-api-access-vjqlf\") pod \"ingress-canary-2kdk2\" (UID: \"6cec1f32-6464-4117-b098-2ed5b3c9cf64\") " pod="openshift-ingress-canary/ingress-canary-2kdk2"
Nov 04 03:17:01.494893 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.494761    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/36af1207-9951-4160-bb36-6dd100a05abb-config-volume\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:01.494893 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.494783    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:01.494893 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:01.494878    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:01.494893 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.494878    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcpp8\" (UniqueName: \"kubernetes.io/projected/36af1207-9951-4160-bb36-6dd100a05abb-kube-api-access-kcpp8\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:01.495168 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.494918    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cec1f32-6464-4117-b098-2ed5b3c9cf64-cert\") pod \"ingress-canary-2kdk2\" (UID: \"6cec1f32-6464-4117-b098-2ed5b3c9cf64\") " pod="openshift-ingress-canary/ingress-canary-2kdk2"
Nov 04 03:17:01.495168 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:01.494934    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls podName:36af1207-9951-4160-bb36-6dd100a05abb nodeName:}" failed. No retries permitted until 2026-11-04 03:17:01.994919396 +0000 UTC m=+33.176417622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls") pod "dns-default-ckjtr" (UID: "36af1207-9951-4160-bb36-6dd100a05abb") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:01.495168 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.495147    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36af1207-9951-4160-bb36-6dd100a05abb-tmp-dir\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:01.495395 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.495371    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/36af1207-9951-4160-bb36-6dd100a05abb-config-volume\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:01.498400 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.498383    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cec1f32-6464-4117-b098-2ed5b3c9cf64-cert\") pod \"ingress-canary-2kdk2\" (UID: \"6cec1f32-6464-4117-b098-2ed5b3c9cf64\") " pod="openshift-ingress-canary/ingress-canary-2kdk2"
Nov 04 03:17:01.558500 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.558468    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcpp8\" (UniqueName: \"kubernetes.io/projected/36af1207-9951-4160-bb36-6dd100a05abb-kube-api-access-kcpp8\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:01.558782 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.558759    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjqlf\" (UniqueName: \"kubernetes.io/projected/6cec1f32-6464-4117-b098-2ed5b3c9cf64-kube-api-access-vjqlf\") pod \"ingress-canary-2kdk2\" (UID: \"6cec1f32-6464-4117-b098-2ed5b3c9cf64\") " pod="openshift-ingress-canary/ingress-canary-2kdk2"
Nov 04 03:17:01.607106 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.607081    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2kdk2"
Nov 04 03:17:01.999993 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:01.999965    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:02.000124 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:02.000078    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:02.000170 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:02.000134    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls podName:36af1207-9951-4160-bb36-6dd100a05abb nodeName:}" failed. No retries permitted until 2026-11-04 03:17:03.000120026 +0000 UTC m=+34.181618288 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls") pod "dns-default-ckjtr" (UID: "36af1207-9951-4160-bb36-6dd100a05abb") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:02.043942 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:17:02.043914    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cec1f32_6464_4117_b098_2ed5b3c9cf64.slice/crio-f218f4e72bd0889609ea84355333941360a1146edf197f800dcfcfce79548005 WatchSource:0}: Error finding container f218f4e72bd0889609ea84355333941360a1146edf197f800dcfcfce79548005: Status 404 returned error can't find the container with id f218f4e72bd0889609ea84355333941360a1146edf197f800dcfcfce79548005
Nov 04 03:17:02.045439 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:02.045418    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2kdk2"]
Nov 04 03:17:02.367644 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:02.367553    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:02.367644 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:02.367579    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:17:02.422770 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:02.422740    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 03:17:02.422963 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:02.422945    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 03:17:02.423020 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:02.422960    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 03:17:02.423020 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:02.423010    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xwzp4\""
Nov 04 03:17:02.423125 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:02.423115    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-mnqmt\""
Nov 04 03:17:02.552627 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:02.552597    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2kdk2" event={"ID":"6cec1f32-6464-4117-b098-2ed5b3c9cf64","Type":"ContainerStarted","Data":"f218f4e72bd0889609ea84355333941360a1146edf197f800dcfcfce79548005"}
Nov 04 03:17:03.007536 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:03.007500    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:03.007706 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:03.007658    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:03.007743 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:03.007723    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls podName:36af1207-9951-4160-bb36-6dd100a05abb nodeName:}" failed. No retries permitted until 2026-11-04 03:17:05.007709132 +0000 UTC m=+36.189207358 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls") pod "dns-default-ckjtr" (UID: "36af1207-9951-4160-bb36-6dd100a05abb") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:03.108554 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:03.108518    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:17:03.108734 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:03.108583    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:03.108734 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:03.108703    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 03:17:03.108870 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:03.108768    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs podName:ae2e6da6-1e6d-4b54-8136-359490273f9c nodeName:}" failed. No retries permitted until 2026-11-04 03:17:35.108751206 +0000 UTC m=+66.290249446 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs") pod "network-metrics-daemon-rz4mh" (UID: "ae2e6da6-1e6d-4b54-8136-359490273f9c") : secret "metrics-daemon-secret" not found
Nov 04 03:17:03.110775 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:03.110749    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:17:03.182150 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:03.182116    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:17:03.201633 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:03.201603    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:03.209676 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:03.209646    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lswx\" (UniqueName: \"kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx\") pod \"network-check-target-5zxn5\" (UID: \"7566445c-b5ac-424d-aa38-a2c5b2e0de21\") " pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:17:03.212158 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:03.212133    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lswx\" (UniqueName: \"kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx\") pod \"network-check-target-5zxn5\" (UID: \"7566445c-b5ac-424d-aa38-a2c5b2e0de21\") " pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:17:03.318470 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:03.318414    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:17:03.346210 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:03.346184    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:03.571425 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:17:03.571389    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39429a51_69a0_4bce_9029_773d5aad2a87.slice/crio-ac88ee6d00f2aeebdd9d27bb0a736060f578e1f50d1f729e5ed5a9ac8c42b99d WatchSource:0}: Error finding container ac88ee6d00f2aeebdd9d27bb0a736060f578e1f50d1f729e5ed5a9ac8c42b99d: Status 404 returned error can't find the container with id ac88ee6d00f2aeebdd9d27bb0a736060f578e1f50d1f729e5ed5a9ac8c42b99d
Nov 04 03:17:03.573453 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:03.573432    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tmrtb"]
Nov 04 03:17:03.694133 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:17:03.694103    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7566445c_b5ac_424d_aa38_a2c5b2e0de21.slice/crio-8bc3d97262e394aa9821b31d05fe1b91f923ddf7466ff19a7aae46cdea52f590 WatchSource:0}: Error finding container 8bc3d97262e394aa9821b31d05fe1b91f923ddf7466ff19a7aae46cdea52f590: Status 404 returned error can't find the container with id 8bc3d97262e394aa9821b31d05fe1b91f923ddf7466ff19a7aae46cdea52f590
Nov 04 03:17:03.695762 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:03.695685    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5zxn5"]
Nov 04 03:17:04.558122 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:04.558083    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5zxn5" event={"ID":"7566445c-b5ac-424d-aa38-a2c5b2e0de21","Type":"ContainerStarted","Data":"8bc3d97262e394aa9821b31d05fe1b91f923ddf7466ff19a7aae46cdea52f590"}
Nov 04 03:17:04.559367 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:04.559334    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tmrtb" event={"ID":"39429a51-69a0-4bce-9029-773d5aad2a87","Type":"ContainerStarted","Data":"ac88ee6d00f2aeebdd9d27bb0a736060f578e1f50d1f729e5ed5a9ac8c42b99d"}
Nov 04 03:17:05.023307 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.023268    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:05.023746 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:05.023407    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:05.023746 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:05.023468    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls podName:36af1207-9951-4160-bb36-6dd100a05abb nodeName:}" failed. No retries permitted until 2026-11-04 03:17:09.023450398 +0000 UTC m=+40.204948644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls") pod "dns-default-ckjtr" (UID: "36af1207-9951-4160-bb36-6dd100a05abb") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:05.747412 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.747380    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vtl99"]
Nov 04 03:17:05.748097 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.748034    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:05.781796 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.781766    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:05.832295 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.832207    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vtl99"]
Nov 04 03:17:05.885525 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.885491    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 03:17:05.885950 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.885923    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 03:17:05.886074 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.885962    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 03:17:05.886074 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.885993    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 03:17:05.889242 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.889222    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jw2mq\""
Nov 04 03:17:05.924042 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.923972    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:05.929722 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.929694    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:05.929836 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.929727    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:05.929836 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.929764    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqp7t\" (UniqueName: \"kubernetes.io/projected/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-api-access-sqp7t\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:05.929974 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.929950    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8830f068-f4c8-492e-8234-4dbf4456f3e5-data-volume\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:05.930034 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:05.930009    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8830f068-f4c8-492e-8234-4dbf4456f3e5-crio-socket\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:06.031084 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.030999    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:06.031084 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.031042    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:06.031084 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.031068    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqp7t\" (UniqueName: \"kubernetes.io/projected/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-api-access-sqp7t\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:06.031630 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.031101    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8830f068-f4c8-492e-8234-4dbf4456f3e5-data-volume\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:06.031630 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.031128    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8830f068-f4c8-492e-8234-4dbf4456f3e5-crio-socket\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:06.031630 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:06.031137    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:06.031630 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:06.031168    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:06.031630 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:06.031218    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.53120083 +0000 UTC m=+37.712699078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:06.031630 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:06.031233    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:06.531227247 +0000 UTC m=+37.712725494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:06.031630 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.031361    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8830f068-f4c8-492e-8234-4dbf4456f3e5-crio-socket\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:06.031630 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.031624    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8830f068-f4c8-492e-8234-4dbf4456f3e5-data-volume\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:06.072604 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.072566    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:06.095282 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.095222    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqp7t\" (UniqueName: \"kubernetes.io/projected/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-api-access-sqp7t\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:06.420149 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.420021    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:06.535164 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.535132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:06.535164 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.535168    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:06.535367 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:06.535276    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:06.535367 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:06.535323    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:07.535310638 +0000 UTC m=+38.716808863 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:06.535660 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:06.535642    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:06.535726 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:06.535685    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:07.53567537 +0000 UTC m=+38.717173596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:06.569591 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.569558    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:06.713756 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.713673    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-llrgs"]
Nov 04 03:17:06.725253 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.725218    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:06.730242 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.730223    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-llrgs"
Nov 04 03:17:06.771527 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.771498    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-llrgs"]
Nov 04 03:17:06.837534 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.837493    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7nf4\" (UniqueName: \"kubernetes.io/projected/99ab9861-53bc-4c46-a53b-f0a0b1a2c53d-kube-api-access-w7nf4\") pod \"migrator-66df5d496b-llrgs\" (UID: \"99ab9861-53bc-4c46-a53b-f0a0b1a2c53d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-llrgs"
Nov 04 03:17:06.863995 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.863965    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:06.864152 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.864070    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 03:17:06.872278 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.872245    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:06.900494 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.900449    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5zf5b\""
Nov 04 03:17:06.938642 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:06.938503    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7nf4\" (UniqueName: \"kubernetes.io/projected/99ab9861-53bc-4c46-a53b-f0a0b1a2c53d-kube-api-access-w7nf4\") pod \"migrator-66df5d496b-llrgs\" (UID: \"99ab9861-53bc-4c46-a53b-f0a0b1a2c53d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-llrgs"
Nov 04 03:17:07.008324 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:07.008294    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7nf4\" (UniqueName: \"kubernetes.io/projected/99ab9861-53bc-4c46-a53b-f0a0b1a2c53d-kube-api-access-w7nf4\") pod \"migrator-66df5d496b-llrgs\" (UID: \"99ab9861-53bc-4c46-a53b-f0a0b1a2c53d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-llrgs"
Nov 04 03:17:07.023639 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:07.023606    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:07.055024 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:07.054996    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-llrgs"
Nov 04 03:17:07.450564 ip-10-0-133-201 kubenswrapper[2574]: W1104 03:17:07.450530    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ab9861_53bc_4c46_a53b_f0a0b1a2c53d.slice/crio-ffe5174236318539aa5d0c7b35f903576317ac68731cc52e02c42d494797a060 WatchSource:0}: Error finding container ffe5174236318539aa5d0c7b35f903576317ac68731cc52e02c42d494797a060: Status 404 returned error can't find the container with id ffe5174236318539aa5d0c7b35f903576317ac68731cc52e02c42d494797a060
Nov 04 03:17:07.452639 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:07.452554    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-llrgs"]
Nov 04 03:17:07.543516 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:07.543479    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:07.543700 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:07.543529    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:07.543700 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:07.543594    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:07.543700 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:07.543673    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:09.543654631 +0000 UTC m=+40.725152875 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:07.543700 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:07.543681    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:07.543931 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:07.543737    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:09.543720724 +0000 UTC m=+40.725218965 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:07.568199 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:07.568160    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-llrgs" event={"ID":"99ab9861-53bc-4c46-a53b-f0a0b1a2c53d","Type":"ContainerStarted","Data":"ffe5174236318539aa5d0c7b35f903576317ac68731cc52e02c42d494797a060"}
Nov 04 03:17:07.570450 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:07.569961    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2kdk2" event={"ID":"6cec1f32-6464-4117-b098-2ed5b3c9cf64","Type":"ContainerStarted","Data":"70b482bfe887c7dad4ee299ec9ce9b8e66d97f42860ce8af684c04ce3cd16c41"}
Nov 04 03:17:07.692688 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:07.692624    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2kdk2" podStartSLOduration=1.794732075 podStartE2EDuration="6.692606352s" totalImagesPullingTime="4.897874277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:07.6919284 +0000 UTC m=+38.873426675" watchObservedRunningTime="2026-11-04 03:17:07.692606352 +0000 UTC m=+38.874104594"
Nov 04 03:17:07.906590 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:07.903860    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:08.600318 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:08.600283    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:09.054463 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:09.054428    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:09.054655 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:09.054592    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:09.054722 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:09.054681    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls podName:36af1207-9951-4160-bb36-6dd100a05abb nodeName:}" failed. No retries permitted until 2026-11-04 03:17:17.054664297 +0000 UTC m=+48.236162548 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls") pod "dns-default-ckjtr" (UID: "36af1207-9951-4160-bb36-6dd100a05abb") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:09.558347 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:09.558311    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:09.558347 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:09.558346    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:09.558573 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:09.558442    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:09.558573 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:09.558462    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:09.558573 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:09.558508    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:13.558496109 +0000 UTC m=+44.739994336 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:09.558573 ip-10-0-133-201 kubenswrapper[2574]: E1104 03:17:09.558521    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:13.558515338 +0000 UTC m=+44.740013563 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:09.576296 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:09.576267    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5zxn5" event={"ID":"7566445c-b5ac-424d-aa38-a2c5b2e0de21","Type":"ContainerStarted","Data":"9096787f83756434d3823c499a0e38c8100edcaa20c1c5155cad08adea82fa44"}
Nov 04 03:17:09.576402 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:09.576343    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:17:09.695737 ip-10-0-133-201 kubenswrapper[2574]: I1104 03:17:09.695686    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5zxn5" podStartSLOduration=35.608436202 podStartE2EDuration="40.695674053s" totalImagesPullingTime="5.087237851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:09.695286092 +0000 UTC m=+40.876784339" watchObservedRunningTime="2026-11-04 03:17:09.695674053 +0000 UTC m=+40.877172298"
Nov 04 03:17:10.122678 ip-10-0-133-201 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 03:17:10.132250 ip-10-0-133-201 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 03:17:10.132418 ip-10-0-133-201 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 03:17:10.132465 ip-10-0-133-201 systemd[1]: kubelet.service: Consumed 2.326s CPU time.
Nov 04 03:17:10.146497 ip-10-0-133-201 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 03:17:10.230573 ip-10-0-133-201 kubenswrapper[4420]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:17:10.230573 ip-10-0-133-201 kubenswrapper[4420]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 03:17:10.230573 ip-10-0-133-201 kubenswrapper[4420]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:17:10.230573 ip-10-0-133-201 kubenswrapper[4420]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:17:10.232606 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232521    4420 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:10.232606 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232606    4420 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232609    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232612    4420 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232615    4420 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232618    4420 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232621    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232624    4420 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232627    4420 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232631    4420 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232633    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232636    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232648    4420 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232651    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232654    4420 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232657    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232660    4420 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232663    4420 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232665    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232668    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:10.232674 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232671    4420 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232674    4420 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232676    4420 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232679    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232682    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232685    4420 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232688    4420 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232690    4420 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232693    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232695    4420 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232698    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232701    4420 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232703    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232706    4420 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232708    4420 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232711    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232713    4420 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232716    4420 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232719    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232721    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:10.233183 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232724    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232727    4420 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232730    4420 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232732    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232734    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232737    4420 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232740    4420 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232743    4420 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232746    4420 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232749    4420 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232751    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232754    4420 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232756    4420 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232761    4420 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232765    4420 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232768    4420 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232772    4420 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232776    4420 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232780    4420 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:10.233675 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232783    4420 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232785    4420 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232788    4420 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232791    4420 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232793    4420 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232797    4420 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232799    4420 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232802    4420 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232804    4420 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232810    4420 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232812    4420 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232815    4420 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232817    4420 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232821    4420 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232824    4420 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232827    4420 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232829    4420 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232832    4420 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232835    4420 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232837    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:10.234213 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232855    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232858    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232861    4420 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232864    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232866    4420 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232869    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232872    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232875    4420 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232877    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232880    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232883    4420 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232885    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232888    4420 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232890    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232893    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232895    4420 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232898    4420 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232900    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232904    4420 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:10.234701 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232906    4420 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232909    4420 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232914    4420 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232917    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232920    4420 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232922    4420 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232925    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.232928    4420 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233678    4420 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233687    4420 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233690    4420 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233693    4420 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233695    4420 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233700    4420 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233705    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233708    4420 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233711    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233714    4420 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233717    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:10.235196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233720    4420 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233723    4420 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233726    4420 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233729    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233732    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233734    4420 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233737    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233740    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233742    4420 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233744    4420 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233747    4420 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233750    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233753    4420 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233755    4420 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233758    4420 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233761    4420 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233764    4420 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233767    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233769    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:10.235670 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233772    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233774    4420 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233777    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233780    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233783    4420 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233785    4420 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233788    4420 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233791    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233793    4420 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233796    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233798    4420 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233801    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233803    4420 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233806    4420 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233808    4420 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233811    4420 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233813    4420 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233816    4420 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233819    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:10.236232 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233821    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233824    4420 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233827    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233830    4420 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233832    4420 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233835    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233838    4420 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233856    4420 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233860    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233864    4420 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233867    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233869    4420 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233872    4420 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233874    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233877    4420 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233880    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233884    4420 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233887    4420 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233890    4420 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233894    4420 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:10.236709 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233896    4420 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233899    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233902    4420 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233904    4420 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233907    4420 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233910    4420 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233912    4420 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233915    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233917    4420 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233920    4420 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233922    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233925    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233928    4420 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233931    4420 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233933    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233936    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233939    4420 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233941    4420 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233944    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233947    4420 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:10.237221 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233949    4420 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233952    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233955    4420 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233957    4420 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233960    4420 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233962    4420 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233965    4420 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233967    4420 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233970    4420 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233973    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233976    4420 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233980    4420 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233983    4420 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233987    4420 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233990    4420 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233993    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.233996    4420 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234512    4420 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234517    4420 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:10.237722 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234521    4420 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234524    4420 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234526    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234529    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234532    4420 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234535    4420 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234537    4420 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234542    4420 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234546    4420 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234549    4420 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234552    4420 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234555    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234558    4420 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234561    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234563    4420 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234566    4420 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234569    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234572    4420 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234574    4420 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:10.238211 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234577    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234579    4420 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234583    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234586    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234588    4420 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234591    4420 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234594    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234598    4420 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234601    4420 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234604    4420 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234607    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234609    4420 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234612    4420 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234614    4420 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234617    4420 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234620    4420 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234622    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234625    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234628    4420 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234630    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:10.238686 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234633    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234635    4420 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234638    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234641    4420 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234644    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234646    4420 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234650    4420 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234652    4420 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234655    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234658    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234660    4420 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234663    4420 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234667    4420 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234670    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234673    4420 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234676    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234686    4420 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234689    4420 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234691    4420 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234696    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:10.239196 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234698    4420 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234701    4420 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234704    4420 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234707    4420 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234710    4420 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234713    4420 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234716    4420 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234719    4420 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234721    4420 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234724    4420 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234726    4420 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234729    4420 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234731    4420 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234734    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234737    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234739    4420 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234742    4420 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234744    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234747    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:10.239714 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234749    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234752    4420 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234754    4420 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234757    4420 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234759    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234762    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234764    4420 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234767    4420 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234769    4420 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234772    4420 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234780    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234783    4420 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234787    4420 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234789    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234792    4420 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234794    4420 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234797    4420 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234800    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234802    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234805    4420 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:10.240198 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234808    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234811    4420 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234814    4420 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234816    4420 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234819    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.234821    4420 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234929    4420 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234937    4420 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234951    4420 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234955    4420 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234960    4420 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234963    4420 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234967    4420 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234971    4420 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234974    4420 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234978    4420 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234981    4420 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234984    4420 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234988    4420 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234991    4420 flags.go:64] FLAG: --cgroup-root=""
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234994    4420 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.234997    4420 flags.go:64] FLAG: --client-ca-file=""
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235000    4420 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 03:17:10.240683 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235009    4420 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235018    4420 flags.go:64] FLAG: --cluster-domain=""
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235021    4420 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235024    4420 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235027    4420 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235030    4420 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235034    4420 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235037    4420 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235042    4420 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235046    4420 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235049    4420 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235052    4420 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235055    4420 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235059    4420 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235062    4420 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235066    4420 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235069    4420 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235072    4420 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235075    4420 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235078    4420 flags.go:64] FLAG: --enable-server="true"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235081    4420 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235089    4420 flags.go:64] FLAG: --event-burst="100"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235092    4420 flags.go:64] FLAG: --event-qps="50"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235095    4420 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 03:17:10.241422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235098    4420 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235101    4420 flags.go:64] FLAG: --eviction-hard=""
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235106    4420 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235108    4420 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235111    4420 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235115    4420 flags.go:64] FLAG: --eviction-soft=""
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235118    4420 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235121    4420 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235124    4420 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235134    4420 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235137    4420 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235140    4420 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235143    4420 flags.go:64] FLAG: --feature-gates=""
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235147    4420 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235150    4420 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235154    4420 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235159    4420 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235162    4420 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235166    4420 flags.go:64] FLAG: --help="false"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235169    4420 flags.go:64] FLAG: --hostname-override="ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235172    4420 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235175    4420 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235178    4420 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 03:17:10.242159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235182    4420 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235185    4420 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235188    4420 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235191    4420 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235194    4420 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235197    4420 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235200    4420 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235204    4420 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235207    4420 flags.go:64] FLAG: --kube-reserved=""
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235210    4420 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235213    4420 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235216    4420 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235219    4420 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235222    4420 flags.go:64] FLAG: --lock-file=""
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235225    4420 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235228    4420 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235231    4420 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235237    4420 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235242    4420 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235250    4420 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235254    4420 flags.go:64] FLAG: --logging-format="text"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235257    4420 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235260    4420 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235263    4420 flags.go:64] FLAG: --manifest-url=""
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235266    4420 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 03:17:10.242757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235272    4420 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235275    4420 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235279    4420 flags.go:64] FLAG: --max-pods="110"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235282    4420 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235285    4420 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235288    4420 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235291    4420 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235294    4420 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235297    4420 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235300    4420 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235307    4420 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235310    4420 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235313    4420 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235316    4420 flags.go:64] FLAG: --pod-cidr=""
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235319    4420 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235322    4420 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235325    4420 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235328    4420 flags.go:64] FLAG: --port="10250"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235331    4420 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235334    4420 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-051361a2477c1b821"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235337    4420 flags.go:64] FLAG: --qos-reserved=""
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235340    4420 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235343    4420 flags.go:64] FLAG: --register-node="true"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235346    4420 flags.go:64] FLAG: --register-with-taints=""
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235350    4420 flags.go:64] FLAG: --registry-burst="10"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235354    4420 flags.go:64] FLAG: --registry-qps="5"
Nov 04 03:17:10.243542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235357    4420 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235367    4420 flags.go:64] FLAG: --reserved-memory=""
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235371    4420 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235374    4420 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235378    4420 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235381    4420 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235385    4420 flags.go:64] FLAG: --runonce="false"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235388    4420 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235391    4420 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235394    4420 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235397    4420 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235400    4420 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235403    4420 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235406    4420 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235410    4420 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235413    4420 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235416    4420 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235419    4420 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235422    4420 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235424    4420 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235428    4420 flags.go:64] FLAG: --system-cgroups=""
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235430    4420 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235435    4420 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235438    4420 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235441    4420 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 03:17:10.244204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235448    4420 flags.go:64] FLAG: --tls-min-version=""
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235451    4420 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235453    4420 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235456    4420 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235459    4420 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235462    4420 flags.go:64] FLAG: --v="2"
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235468    4420 flags.go:64] FLAG: --version="false"
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235472    4420 flags.go:64] FLAG: --vmodule=""
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235476    4420 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.235487    4420 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235663    4420 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235667    4420 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235670    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235675    4420 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235678    4420 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235681    4420 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235684    4420 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235687    4420 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235689    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235692    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235694    4420 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235697    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:10.244804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235700    4420 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235703    4420 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235705    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235708    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235710    4420 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235719    4420 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235722    4420 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235724    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235727    4420 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235730    4420 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235734    4420 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235737    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235740    4420 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235743    4420 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235746    4420 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235749    4420 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235753    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235755    4420 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235758    4420 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:10.245400 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235761    4420 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235770    4420 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235773    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235775    4420 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235781    4420 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235784    4420 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235786    4420 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235789    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235791    4420 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235794    4420 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235797    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235800    4420 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235802    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235805    4420 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235807    4420 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235810    4420 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235813    4420 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235815    4420 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235818    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235820    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:10.245879 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235823    4420 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235825    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235828    4420 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235830    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235833    4420 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235835    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235838    4420 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235855    4420 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235858    4420 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235864    4420 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235866    4420 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235869    4420 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235871    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235874    4420 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235884    4420 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235887    4420 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235892    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235895    4420 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235897    4420 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235900    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:10.246394 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235902    4420 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235905    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235907    4420 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235910    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235912    4420 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235915    4420 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235918    4420 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235922    4420 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235925    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235928    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235931    4420 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235934    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235937    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235939    4420 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235942    4420 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235944    4420 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235947    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235949    4420 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235952    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:10.246993 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235954    4420 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235957    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235961    4420 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235963    4420 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235966    4420 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235968    4420 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235971    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235974    4420 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235982    4420 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235987    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235989    4420 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235992    4420 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235994    4420 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.235997    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236000    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236002    4420 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:10.247460 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.236029    4420 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236076    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236080    4420 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236083    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236086    4420 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236088    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236091    4420 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236094    4420 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236097    4420 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236101    4420 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236105    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236108    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236111    4420 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236114    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236116    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236119    4420 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236121    4420 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236124    4420 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236127    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:10.247936 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236131    4420 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236133    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236136    4420 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236138    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236141    4420 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236144    4420 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236153    4420 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236156    4420 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236159    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236162    4420 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236164    4420 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236167    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236169    4420 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236172    4420 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236175    4420 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236178    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236180    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236183    4420 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236185    4420 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:10.248501 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236188    4420 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236190    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236193    4420 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236196    4420 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236198    4420 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236201    4420 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236203    4420 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236206    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236209    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236211    4420 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236214    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236217    4420 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236219    4420 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236223    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236226    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236228    4420 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236231    4420 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236233    4420 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236236    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:10.248990 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236240    4420 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236249    4420 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236254    4420 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236258    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236261    4420 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236264    4420 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236266    4420 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236269    4420 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236272    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236275    4420 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236277    4420 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236280    4420 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236282    4420 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236285    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236288    4420 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236290    4420 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236293    4420 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236295    4420 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236298    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:10.249492 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236301    4420 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236303    4420 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236306    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236309    4420 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236311    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236314    4420 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236316    4420 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236320    4420 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236322    4420 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236325    4420 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236327    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236330    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236332    4420 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236337    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236339    4420 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236347    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236351    4420 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236353    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236356    4420 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236358    4420 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:10.260144 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236361    4420 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236363    4420 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236366    4420 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236368    4420 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236371    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236374    4420 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236377    4420 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236380    4420 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236382    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236385    4420 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.236387    4420 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.240484    4420 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.240502    4420 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.240519    4420 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.240524    4420 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240583    4420 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240588    4420 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240591    4420 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240595    4420 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240598    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:10.261003 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240600    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240603    4420 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240606    4420 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240609    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240612    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240614    4420 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240617    4420 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240620    4420 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240623    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240626    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240629    4420 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240631    4420 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240634    4420 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240636    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240639    4420 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240642    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240645    4420 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240650    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240653    4420 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240656    4420 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:10.261735 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240658    4420 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240661    4420 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240664    4420 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240667    4420 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240675    4420 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240679    4420 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240681    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240684    4420 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240687    4420 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240690    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240692    4420 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240695    4420 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240698    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240701    4420 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240704    4420 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240706    4420 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240709    4420 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240720    4420 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240723    4420 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240726    4420 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:10.262250 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240729    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240731    4420 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240734    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240737    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240739    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240742    4420 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240744    4420 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240747    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240750    4420 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240753    4420 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240755    4420 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240758    4420 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240761    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240764    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240767    4420 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240769    4420 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240772    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240780    4420 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240784    4420 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:10.262730 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240786    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240789    4420 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240791    4420 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240794    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240796    4420 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240799    4420 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240802    4420 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240804    4420 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240807    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240809    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240812    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240814    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240817    4420 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240819    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240822    4420 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240825    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240827    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240830    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240832    4420 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:10.263258 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240835    4420 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240837    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240852    4420 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240855    4420 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240858    4420 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240860    4420 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240863    4420 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240866    4420 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240870    4420 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240874    4420 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240877    4420 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240881    4420 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240890    4420 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240893    4420 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240896    4420 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240898    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240901    4420 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240903    4420 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240906    4420 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:10.263828 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240909    4420 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240911    4420 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240914    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.240916    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.240944    4420 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241805    4420 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241820    4420 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241823    4420 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241826    4420 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241828    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241831    4420 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241834    4420 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241837    4420 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241853    4420 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241857    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:10.264515 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241866    4420 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241869    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241872    4420 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241875    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241878    4420 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241881    4420 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241883    4420 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241886    4420 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241890    4420 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241893    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241895    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241910    4420 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241913    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241919    4420 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241922    4420 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241924    4420 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241927    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241930    4420 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241933    4420 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:10.264910 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241936    4420 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241939    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241941    4420 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241944    4420 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241947    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241950    4420 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241955    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241958    4420 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241961    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241964    4420 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241967    4420 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241971    4420 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241975    4420 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241978    4420 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241981    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241983    4420 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241986    4420 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241989    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241992    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:10.265368 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.241998    4420 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242001    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242003    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242006    4420 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242009    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242012    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242025    4420 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242028    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242031    4420 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242033    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242036    4420 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242039    4420 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242041    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242047    4420 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242050    4420 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242053    4420 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242058    4420 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242061    4420 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242065    4420 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:10.265870 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242069    4420 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242072    4420 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242076    4420 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242079    4420 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242081    4420 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242084    4420 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242090    4420 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242092    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242095    4420 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242098    4420 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242101    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242104    4420 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242107    4420 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242110    4420 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242113    4420 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242116    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242119    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242122    4420 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242127    4420 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242130    4420 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:10.266336 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242145    4420 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242148    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242151    4420 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242153    4420 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242156    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242159    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242162    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242165    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242167    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242172    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242175    4420 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242180    4420 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242183    4420 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242186    4420 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242188    4420 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242191    4420 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242194    4420 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242196    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:10.266804 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242199    4420 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.242235    4420 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242317    4420 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242322    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242325    4420 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242328    4420 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242331    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242334    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242337    4420 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242339    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242342    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242345    4420 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242350    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242353    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242356    4420 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242369    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:17:10.267415 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242372    4420 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242374    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242377    4420 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242380    4420 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242383    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242386    4420 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242389    4420 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242392    4420 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242396    4420 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242401    4420 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242405    4420 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242410    4420 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242413    4420 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242415    4420 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242418    4420 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242421    4420 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242424    4420 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242427    4420 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242429    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242432    4420 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:17:10.268783 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242435    4420 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:17:10.267524 ip-10-0-133-201 systemd[1]: Started Kubernetes Kubelet.
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242440    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242443    4420 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242446    4420 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242449    4420 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242451    4420 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242454    4420 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242457    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242460    4420 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242462    4420 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242465    4420 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242468    4420 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242481    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242486    4420 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242491    4420 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242505    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242508    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242511    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242513    4420 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242516    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:17:10.269867 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242520    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242523    4420 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242526    4420 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242528    4420 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242531    4420 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242534    4420 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242539    4420 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242542    4420 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242545    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242548    4420 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242550    4420 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242553    4420 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242556    4420 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242558    4420 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242561    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242564    4420 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242567    4420 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242569    4420 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242572    4420 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:17:10.270456 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242577    4420 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242580    4420 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242583    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242585    4420 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242588    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242591    4420 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242603    4420 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242608    4420 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242611    4420 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242614    4420 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242617    4420 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242619    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242734    4420 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242779    4420 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242786    4420 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242791    4420 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242796    4420 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242802    4420 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242808    4420 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:17:10.270956 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242813    4420 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242818    4420 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242822    4420 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242827    4420 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242832    4420 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242859    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242887    4420 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242892    4420 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242896    4420 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242901    4420 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242905    4420 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242909    4420 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242942    4420 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:10.242948    4420 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.243139    4420 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.244159    4420 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.244211    4420 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.244384    4420 server.go:1014] "Starting client certificate rotation"
Nov 04 03:17:10.271687 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.244516    4420 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 03:17:10.272509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.244559    4420 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 03:11:29 +0000 UTC" deadline="2028-06-06 19:34:48.056181534 +0000 UTC"
Nov 04 03:17:10.272509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.244586    4420 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13936h17m37.811597591s"
Nov 04 03:17:10.272509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.244831    4420 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:17:10.272509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.245873    4420 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:17:10.272509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.247317    4420 log.go:25] "Validated CRI v1 runtime API"
Nov 04 03:17:10.272509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.248874    4420 log.go:25] "Validated CRI v1 image API"
Nov 04 03:17:10.272509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.249598    4420 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 03:17:10.272509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.251078    4420 fs.go:135] Filesystem UUIDs: map[6ca3d19c-ee97-4728-8a18-cc6b7f7ef1b5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a6a02d61-27a8-4448-bcfa-8fc8c9ef354a:/dev/nvme0n1p3]
Nov 04 03:17:10.272938 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.251100    4420 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/226a4227a8a92d34d1406056e151c3db79dbb29244396dd744f6829da5e23522/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/226a4227a8a92d34d1406056e151c3db79dbb29244396dd744f6829da5e23522/userdata/shm major:0 minor:64 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/62a22db6e28f2b130c91e53743021384e0ebcf95cfb58541d96f8a57750c4b2d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/62a22db6e28f2b130c91e53743021384e0ebcf95cfb58541d96f8a57750c4b2d/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/649cc2ad8030b26a82eee98ded3ee17b4e15056a72a7fd0753a3945628043328/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/649cc2ad8030b26a82eee98ded3ee17b4e15056a72a7fd0753a3945628043328/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6fb613bc7c958935a8d23066b5bc51099623fab47f72c4dd27825c6579aba8b8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6fb613bc7c958935a8d23066b5bc51099623fab47f72c4dd27825c6579aba8b8/userdata/shm major:0 minor:100 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/824d2cbbc83cbdfeeb4918c559ec65a0d7f6d1274543f433770e0ab40fd6126c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/824d2cbbc83cbdfeeb4918c559ec65a0d7f6d1274543f433770e0ab40fd6126c/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8bc3d97262e394aa9821b31d05fe1b91f923ddf7466ff19a7aae46cdea52f590/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8bc3d97262e394aa9821b31d05fe1b91f923ddf7466ff19a7aae46cdea52f590/userdata/shm major:0 minor:210 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8dc619aefb0904ee71ece245244cf08e87f4e4a160db525930f37e2107a07489/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8dc619aefb0904ee71ece245244cf08e87f4e4a160db525930f37e2107a07489/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ac88ee6d00f2aeebdd9d27bb0a736060f578e1f50d1f729e5ed5a9ac8c42b99d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ac88ee6d00f2aeebdd9d27bb0a736060f578e1f50d1f729e5ed5a9ac8c42b99d/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c4429e7010d037d1175d80fa1f248377ab4d6a97caf4ef8888ab956eb40d27bf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c4429e7010d037d1175d80fa1f248377ab4d6a97caf4ef8888ab956eb40d27bf/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cee6bfcaf1bd44ae12c618ae6c1e053472d1563db85a7d7396a7ecd8d92e5737/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cee6bfcaf1bd44ae12c618ae6c1e053472d1563db85a7d7396a7ecd8d92e5737/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d67c059c089997458d92b6729f9515c79085d26cd13de7bcc543b3ba17a4497e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d67c059c089997458d92b6729f9515c79085d26cd13de7bcc543b3ba17a4497e/userdata/shm major:0 minor:83 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ef921ad20b1e70cfaac7f27cd34d3b0667f758fa4e4241d6be625e40cd6dc350/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ef921ad20b1e70cfaac7f27cd34d3b0667f758fa4e4241d6be625e40cd6dc350/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f218f4e72bd0889609ea84355333941360a1146edf197f800dcfcfce79548005/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f218f4e72bd0889609ea84355333941360a1146edf197f800dcfcfce79548005/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ffe5174236318539aa5d0c7b35f903576317ac68731cc52e02c42d494797a060/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ffe5174236318539aa5d0c7b35f903576317ac68731cc52e02c42d494797a060/userdata/shm major:0 minor:219 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0599c4e4-3db5-42b2-bd74-2c874ab74383/volumes/kubernetes.io~projected/kube-api-access-rzzhc:{mountpoint:/var/lib/kubelet/pods/0599c4e4-3db5-42b2-bd74-2c874ab74383/volumes/kubernetes.io~projected/kube-api-access-rzzhc major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/36af1207-9951-4160-bb36-6dd100a05abb/volumes/kubernetes.io~projected/kube-api-access-kcpp8:{mountpoint:/var/lib/kubelet/pods/36af1207-9951-4160-bb36-6dd100a05abb/volumes/kubernetes.io~projected/kube-api-access-kcpp8 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/39429a51-69a0-4bce-9029-773d5aad2a87/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/39429a51-69a0-4bce-9029-773d5aad2a87/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/39640be1-9711-4cef-a9a2-430ea40c7c18/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/39640be1-9711-4cef-a9a2-430ea40c7c18/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/39640be1-9711-4cef-a9a2-430ea40c7c18/volumes/kubernetes.io~projected/kube-api-access-tsm6h:{mountpoint:/var/lib/kubelet/pods/39640be1-9711-4cef-a9a2-430ea40c7c18/volumes/kubernetes.io~projected/kube-api-access-tsm6h major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/39640be1-9711-4cef-a9a2-430ea40c7c18/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/39640be1-9711-4cef-a9a2-430ea40c7c18/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3aef6eaf-2dde-4706-bcf9-77bf9c447028/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/3aef6eaf-2dde-4706-bcf9-77bf9c447028/volumes/kubernetes.io~secret/agent-certs major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/50f36ad6-52cd-4239-94eb-22b8f3de2a26/volumes/kubernetes.io~projected/kube-api-access-grkj6:{mountpoint:/var/lib/kubelet/pods/50f36ad6-52cd-4239-94eb-22b8f3de2a26/volumes/kubernetes.io~projected/kube-api-access-grkj6 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6cec1f32-6464-4117-b098-2ed5b3c9cf64/volumes/kubernetes.io~projected/kube-api-access-vjqlf:{mountpoint:/var/lib/kubelet/pods/6cec1f32-6464-4117-b098-2ed5b3c9cf64/volumes/kubernetes.io~projected/kube-api-access-vjqlf major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6cec1f32-6464-4117-b098-2ed5b3c9cf64/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/6cec1f32-6464-4117-b098-2ed5b3c9cf64/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7566445c-b5ac-424d-aa38-a2c5b2e0de21/volumes/kubernetes.io~projected/kube-api-access-5lswx:{mountpoint:/var/lib/kubelet/pods/7566445c-b5ac-424d-aa38-a2c5b2e0de21/volumes/kubernetes.io~projected/kube-api-access-5lswx major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7586e6a7-b118-4cef-9c67-56e5a5d4eb04/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/7586e6a7-b118-4cef-9c67-56e5a5d4eb04/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7586e6a7-b118-4cef-9c67-56e5a5d4eb04/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/7586e6a7-b118-4cef-9c67-56e5a5d4eb04/volumes/kubernetes.io~empty-dir/tmp major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7586e6a7-b118-4cef-9c67-56e5a5d4eb04/volumes/kubernetes.io~projected/kube-api-access-s9499:{mountpoint:/var/lib/kubelet/pods/7586e6a7-b118-4cef-9c67-56e5a5d4eb04/volumes/kubernetes.io~projected/kube-api-access-s9499 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8830f068-f4c8-492e-8234-4dbf4456f3e5/volumes/kubernetes.io~projected/kube-api-access-sqp7t:{mountpoint:/var/lib/kubelet/pods/8830f068-f4c8-492e-8234-4dbf4456f3e5/volumes/kubernetes.io~projected/kube-api-access-sqp7t major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99ab9861-53bc-4c46-a53b-f0a0b1a2c53d/volumes/kubernetes.io~projected/kube-api-access-w7nf4:{mountpoint:/var/lib/kubelet/pods/99ab9861-53bc-4c46-a53b-f0a0b1a2c53d/volumes/kubernetes.io~projected/kube-api-access-w7nf4 major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae2e6da6-1e6d-4b54-8136-359490273f9c/volumes/kubernetes.io~projected/kube-api-access-wz7p8:{mountpoint:/var/lib/kubelet/pods/ae2e6da6-1e6d-4b54-8136-359490273f9c/volumes/kubernetes.io~projected/kube-api-access-wz7p8 major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd/volumes/kubernetes.io~projected/kube-api-access-5st2r:{mountpoint:/var/lib/kubelet/pods/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd/volumes/kubernetes.io~projected/kube-api-access-5st2r major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e0c7e7cd-abcd-429b-b556-947ac5d9402a/volumes/kubernetes.io~projected/kube-api-access-22q58:{mountpoint:/var/lib/kubelet/pods/e0c7e7cd-abcd-429b-b556-947ac5d9402a/volumes/kubernetes.io~projected/kube-api-access-22q58 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e1685aa8-41b4-4899-ad2d-6e03e2969d72/volumes/kubernetes.io~projected/kube-api-access-bl5th:{mountpoint:/var/lib/kubelet/pods/e1685aa8-41b4-4899-ad2d-6e03e2969d72/volumes/kubernetes.io~projected/kube-api-access-bl5th major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e1f6fbb1-af6e-43e4-a671-68a6c3691463/volumes/kubernetes.io~projected/kube-api-access-8dnnx:{mountpoint:/var/lib/kubelet/pods/e1f6fbb1-af6e-43e4-a671-68a6c3691463/volumes/kubernetes.io~projected/kube-api-access-8dnnx major:0 minor:53 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/893a36b051016046480a68dda129743fb77e8afecec7aa3e9ca7fbb2b6de54d3/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/b2cb16b347fba1811d09ebbc3d7ee8a12e6054bf93bd926715f739aeebf5bb23/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/f3cd9b3a81be10a27a6861879521d4eb71fe0eed03b99eb6da2b42a232450af4/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/0888ac1be5b73f9ebc976003d13c4a89da2e8ce568b18ad17d7e2ffc328a5f21/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-117:{mountpoint:/var/lib/containers/storage/overlay/5eeb4bd638ab4e7f577b6b6d493b0dd40194df8c45e1b671ba12542e7e708805/merged major:0 minor:117 fsType:overlay blockSize:0} overlay_0-122:{mountpoint:/var/lib/containers/storage/overlay/c4bac652c4fcd15f8d28a07de57baa82d1c085f07255b907c2c504aebb2dd4a5/merged major:0 minor:122 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/4987d26aa5e9d4464673bcd19c7c40bb4d6365056b36df41c80a40b7cf1e1849/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/a0c01874c3f30f9dc991d6ae15e7d2680f16b0b5af3f42c6ffb43be45e384793/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/f9493092628b26f8279aedfb0f4f7b0bdeb54a98e71fc19f7a0f30803e715679/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/2414af29767fbf94e04ee74d4eca3dea35750b413bacbafb1a1e61ecf070acf1/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/a1346f5be6ce3d6b041d6b934290799f5eb7587234348fe0cd875caab06d6308/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/d52f4483c636e28addada378fbdf0ee9544b183dbeb591d9bf3b4a11f85f48ba/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/a5b35ecf98a49592e75afd43d3572f548f0f0c094b36dc80628451a70780355a/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-158:{mountpoint:/var/lib/containers/storage/overlay/c44021546287f9a230ae36c4eb9ef5c75c68056755ad813da9e58862813f2c7a/merged major:0 minor:158 fsType:overlay blockSize:0} overlay_0-161:{mountpoint:/var/lib/containers/storage/overlay/8da8cce27fe6405e8df4bd8714e3421c8c54738399d6a1a88fba60220e0044da/merged major:0 minor:161 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/fee50f8064a1f2434cda06667256bad3c4eb357c694ef45e3e467f90e002a92a/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/ef3f30575e80548ae8fa1ba7deebb0096357f7e3bb59d611e92a01b5d06c8612/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/946471064f057b3e352aa4b7ba556ec645c1b8141db61d643c239f35c4c4edd8/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/106ab0febe40514e1613c1582e4004df9ed148223448c77fd47a5ce3880c7773/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/afa163f56a101e88e3796cb76de7f9222efacaa892df9419a9c388d9fd451c18/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/9040b165d5787fa81b7150ffaceb76ef1fe4bdb31a5fd246788d714e66289f52/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-212:{mountpoint:/var/lib/containers/storage/overlay/defc4fbda9c8cfafa5e0dc2910f01628360b1d8d12a3c6c860634d92aae857eb/merged major:0 minor:212 fsType:overlay blockSize:0} overlay_0-214:{mountpoint:/var/lib/containers/storage/overlay/661780b317fcb2c514208fd51f4c94c62855c87eee731bfaea466ea7abadacc6/merged major:0 minor:214 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/1f8a4d3ef7e317e5962d6188cbc82c684ad3cd7ce8c0e40f25ca60f5b5e21e1a/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-226:{mountpoint:/var/lib/containers/storage/overlay/13aea8ac494d9b973b1fc2714d68d2039c3993efabaf573209961968899102c7/merged major:0 minor:226 fsType:overlay blockSize:0} overlay_0-228:{mountpoint:/var/lib/containers/storage/overlay/810a024dffa1e30316cf1a06554ec5900d8eccbcf7d70fda5b216f1f53aee82a/merged major:0 minor:228 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/28bdd9800311f001891e8648ce42e0f7f482289206fb2ce169a20a2b66b1ba1e/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/71026338d261b004ea3db419a78cb9baab5b8bec54ced9d43aa64e9dde506a37/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/48be91c279fdeb2ca17341da990422233411181f92574cddd6570627da00276e/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/62f494b815fabfc49e000127049abe8e7f072c68f88f5523722533e9993659d5/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/0815cefe581615433514095e5bfebd8b775aaef391ca0c2f9f06e87e474a21e6/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/3c9a5c012d73d5abb586ef6ed6406a110e526ff4d6259bec42a2130e2497eb41/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/25a1750e67a273be3c8adac13886ff32f4357b4f2c07389b4be3212cf933ec5d/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/cf3e0aea73f4d7a82aab0a7b78c328bc02e1ea69006386f7e6c503682a7273bd/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-85:{mountpoint:/var/lib/containers/storage/overlay/d1c264ac1a7f0c5134b209a21bcda85d9fb5584a3c982473d13dd09ce72d1c70/merged major:0 minor:85 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/eef1ac47777102470209889983d61f50d464ae3d3ece1dd786df1fc433277b19/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/7f6c9a30ef6c59bf45e95e94f91a64cbbda13dc151acbc5188923062fa5866f6/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/d25780f84e2ffec4c3b0cd1752b639ae20d29f72295360f286dab692ff65f0f5/merged major:0 minor:96 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/ecd087709f17cb319f6d9b454428f2c795fc03803d1b8e743b345394038d9e51/merged major:0 minor:98 fsType:overlay blockSize:0}]
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.257072    4420 manager.go:217] Machine: {Timestamp:2026-11-04 03:17:10.256598309 +0000 UTC m=+0.071389578 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099349 MemoryCapacity:33164328960 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a56e9b9ee18983e02fc8c56b6dd47 SystemUUID:ec2a56e9-b9ee-1898-3e02-fc8c56b6dd47 BootID:3249a06b-f6ac-4685-b117-019264acedc7 Filesystems:[{Device:/run/containers/storage/overlay-containers/cee6bfcaf1bd44ae12c618ae6c1e053472d1563db85a7d7396a7ecd8d92e5737/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/6cec1f32-6464-4117-b098-2ed5b3c9cf64/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-212 DeviceMajor:0 DeviceMinor:212 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7586e6a7-b118-4cef-9c67-56e5a5d4eb04/volumes/kubernetes.io~projected/kube-api-access-s9499 DeviceMajor:0 DeviceMinor:57 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-122 DeviceMajor:0 DeviceMinor:122 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f218f4e72bd0889609ea84355333941360a1146edf197f800dcfcfce79548005/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-214 DeviceMajor:0 DeviceMinor:214 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd/volumes/kubernetes.io~projected/kube-api-access-5st2r DeviceMajor:0 DeviceMinor:54 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-158 DeviceMajor:0 DeviceMinor:158 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6cec1f32-6464-4117-b098-2ed5b3c9cf64/volumes/kubernetes.io~projected/kube-api-access-vjqlf DeviceMajor:0 DeviceMinor:201 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/7586e6a7-b118-4cef-9c67-56e5a5d4eb04/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:51 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ae2e6da6-1e6d-4b54-8136-359490273f9c/volumes/kubernetes.io~projected/kube-api-access-wz7p8 DeviceMajor:0 DeviceMinor:55 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/226a4227a8a92d34d1406056e151c3db79dbb29244396dd744f6829da5e23522/userdata/shm DeviceMajor:0 DeviceMinor:64 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ef921ad20b1e70cfaac7f27cd34d3b0667f758fa4e4241d6be625e40cd6dc350/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-161 DeviceMajor:0 DeviceMinor:161 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8dc619aefb0904ee71ece245244cf08e87f4e4a160db525930f37e2107a07489/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/50f36ad6-52cd-4239-94eb-22b8f3de2a26/volumes/kubernetes.io~projected/kube-api-access-grkj6 DeviceMajor:0 DeviceMinor:58 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ac88ee6d00f2aeebdd9d27bb0a736060f578e1f50d1f729e5ed5a9ac8c42b99d/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/99ab9861-53bc-4c46-a53b-f0a0b1a2c53d/volumes/kubernetes.io~projected/kube-api-access-w7nf4 DeviceMajor:0 DeviceMinor:224 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e0c7e7cd-abcd-429b-b556-947ac5d9402a/volumes/kubernetes.io~projected/kube-api-access-22q58 DeviceMajor:0 DeviceMinor:59 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/39640be1-9711-4cef-a9a2-430ea40c7c18/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:60 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/39640be1-9711-4cef-a9a2-430ea40c7c18/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e1f6fbb1-af6e-43e4-a671-68a6c3691463/volumes/kubernetes.io~projected/kube-api-access-8dnnx DeviceMajor:0 DeviceMinor:53 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/62a22db6e28f2b130c91e53743021384e0ebcf95cfb58541d96f8a57750c4b2d/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6fb613bc7c958935a8d23066b5bc51099623fab47f72c4dd27825c6579aba8b8/userdata/shm DeviceMajor:0 DeviceMinor:100 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-117 DeviceMajor:0 DeviceMinor:117 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/36af1207-9951-4160-bb36-6dd100a05abb/volumes/kubernetes.io~projected/kube-api-access-kcpp8 DeviceMajor:0 DeviceMinor:197 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-226 DeviceMajor:0 DeviceMinor:226 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7586e6a7-b118-4cef-9c67-56e5a5d4eb04/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:52 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/0599c4e4-3db5-42b2-bd74-2c874ab74383/volumes/kubernetes.io~projected/kube-api-access-rzzhc DeviceMajor:0 DeviceMinor:56 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ffe5174236318539aa5d0c7b35f903576317ac68731cc52e02c42d494797a060/userdata/shm DeviceMajor:0 DeviceMinor:219 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c4429e7010d037d1175d80fa1f248377ab4d6a97caf4ef8888ab956eb40d27bf/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7566445c-b5ac-424d-aa38-a2c5b2e0de21/volumes/kubernetes.io~projected/kube-api-access-5lswx DeviceMajor:0 DeviceMinor:207 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8bc3d97262e394aa9821b31d05fe1b91f923ddf7466ff19a7aae46cdea52f590/userdata/shm DeviceMajor:0 DeviceMinor:210 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/824d2cbbc83cbdfeeb4918c559ec65a0d7f6d1274543f433770e0ab40fd6126c/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-85 DeviceMajor:0 DeviceMinor:85 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-228 DeviceMajor:0 DeviceMinor:228 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3aef6eaf-2dde-4706-bcf9-77bf9c447028/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:61 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/39640be1-9711-4cef-a9a2-430ea40c7c18/volumes/kubernetes.io~projected/kube-api-access-tsm6h DeviceMajor:0 DeviceMinor:62 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/e1685aa8-41b4-4899-ad2d-6e03e2969d72/volumes/kubernetes.io~projected/kube-api-access-bl5th DeviceMajor:0 DeviceMinor:63 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/649cc2ad8030b26a82eee98ded3ee17b4e15056a72a7fd0753a3945628043328/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d67c059c089997458d92b6729f9515c79085d26cd13de7bcc543b3ba17a4497e/userdata/shm DeviceMajor:0 DeviceMinor:83 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/39429a51-69a0-4bce-9029-773d5aad2a87/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:206 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/8830f068-f4c8-492e-8234-4dbf4456f3e5/volumes/kubernetes.io~projected/kube-api-access-sqp7t DeviceMajor:0 DeviceMinor:216 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:8bc3d97262e394a MacAddress:2a:70:58:b7:1a:17 Speed:10000 Mtu:8901} {Name:ac88ee6d00f2aee MacAddress:9e:16:f3:d4:c0:91 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:e2:99:98:43 Speed:0 Mtu:9001} {Name:br-int MacAddress:52:f8:e7:f6:47:48 Speed:0 Mtu:8901} {Name:ens5 MacAddress:06:ff:e2:99:98:43 Speed:0 Mtu:9001} {Name:f218f4e72bd0889 MacAddress:ba:50:98:f5:6c:f0 Speed:10000 Mtu:8901} {Name:ffe517423631853 MacAddress:72:5d:06:44:3d:c4 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:c2:f8:ff:4c:d5:c2 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:2e:47:6b:c9:05:3a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164328960 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.257309    4420 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.257355    4420 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.257701    4420 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.257718    4420 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-201.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.257880    4420 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.257894    4420 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.257904    4420 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.257918    4420 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.257994    4420 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.258181    4420 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.258334    4420 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.258371    4420 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.258418    4420 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.258432    4420 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.258495    4420 kubelet.go:405] "Adding apiserver pod source"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.258508    4420 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.260129    4420 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.260154    4420 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.260692    4420 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263141    4420 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263151    4420 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263245    4420 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263436    4420 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263462    4420 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263547    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263560    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263566    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263571    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263577    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263583    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263589    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263594    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263600    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263606    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263615    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263625    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263650    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.263655    4420 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.266976    4420 server.go:1263] "Started kubelet"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.267083    4420 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.267093    4420 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.267197    4420 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.267706    4420 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.268795    4420 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.271292    4420 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.271804    4420 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.271818    4420 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.271815    4420 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.271948    4420 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.272332    4420 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:11:31 +0000 UTC" deadline="2028-04-07 03:23:53.817706233 +0000 UTC"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.272365    4420 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12480h6m43.545342903s"
Nov 04 03:17:10.273166 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:10.272784    4420 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273424    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0599c4e4-3db5-42b2-bd74-2c874ab74383" volumeName="kubernetes.io/projected/0599c4e4-3db5-42b2-bd74-2c874ab74383-kube-api-access-rzzhc" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273455    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="39640be1-9711-4cef-a9a2-430ea40c7c18" volumeName="kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-ovnkube-config" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273469    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3aef6eaf-2dde-4706-bcf9-77bf9c447028" volumeName="kubernetes.io/configmap/3aef6eaf-2dde-4706-bcf9-77bf9c447028-konnectivity-ca" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273485    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0599c4e4-3db5-42b2-bd74-2c874ab74383" volumeName="kubernetes.io/configmap/0599c4e4-3db5-42b2-bd74-2c874ab74383-serviceca" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273496    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3aef6eaf-2dde-4706-bcf9-77bf9c447028" volumeName="kubernetes.io/secret/3aef6eaf-2dde-4706-bcf9-77bf9c447028-agent-certs" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273508    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7586e6a7-b118-4cef-9c67-56e5a5d4eb04" volumeName="kubernetes.io/empty-dir/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-tmp" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273521    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="39640be1-9711-4cef-a9a2-430ea40c7c18" volumeName="kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-env-overrides" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273532    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="39640be1-9711-4cef-a9a2-430ea40c7c18" volumeName="kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273554    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6cec1f32-6464-4117-b098-2ed5b3c9cf64" volumeName="kubernetes.io/secret/6cec1f32-6464-4117-b098-2ed5b3c9cf64-cert" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273568    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8830f068-f4c8-492e-8234-4dbf4456f3e5" volumeName="kubernetes.io/empty-dir/8830f068-f4c8-492e-8234-4dbf4456f3e5-data-volume" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273580    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8830f068-f4c8-492e-8234-4dbf4456f3e5" volumeName="kubernetes.io/projected/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-api-access-sqp7t" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273591    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99ab9861-53bc-4c46-a53b-f0a0b1a2c53d" volumeName="kubernetes.io/projected/99ab9861-53bc-4c46-a53b-f0a0b1a2c53d-kube-api-access-w7nf4" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273608    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" volumeName="kubernetes.io/projected/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-kube-api-access-5st2r" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273620    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="36af1207-9951-4160-bb36-6dd100a05abb" volumeName="kubernetes.io/empty-dir/36af1207-9951-4160-bb36-6dd100a05abb-tmp-dir" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273637    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="39640be1-9711-4cef-a9a2-430ea40c7c18" volumeName="kubernetes.io/projected/39640be1-9711-4cef-a9a2-430ea40c7c18-kube-api-access-tsm6h" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273650    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6cec1f32-6464-4117-b098-2ed5b3c9cf64" volumeName="kubernetes.io/projected/6cec1f32-6464-4117-b098-2ed5b3c9cf64-kube-api-access-vjqlf" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273662    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7566445c-b5ac-424d-aa38-a2c5b2e0de21" volumeName="kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273675    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7586e6a7-b118-4cef-9c67-56e5a5d4eb04" volumeName="kubernetes.io/projected/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-kube-api-access-s9499" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273687    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" volumeName="kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273700    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0c7e7cd-abcd-429b-b556-947ac5d9402a" volumeName="kubernetes.io/configmap/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-daemon-config" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273712    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0c7e7cd-abcd-429b-b556-947ac5d9402a" volumeName="kubernetes.io/projected/e0c7e7cd-abcd-429b-b556-947ac5d9402a-kube-api-access-22q58" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273726    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="36af1207-9951-4160-bb36-6dd100a05abb" volumeName="kubernetes.io/configmap/36af1207-9951-4160-bb36-6dd100a05abb-config-volume" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273738    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" volumeName="kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cni-binary-copy" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273750    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" volumeName="kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273763    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1685aa8-41b4-4899-ad2d-6e03e2969d72" volumeName="kubernetes.io/projected/e1685aa8-41b4-4899-ad2d-6e03e2969d72-kube-api-access-bl5th" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273776    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1f6fbb1-af6e-43e4-a671-68a6c3691463" volumeName="kubernetes.io/projected/e1f6fbb1-af6e-43e4-a671-68a6c3691463-kube-api-access-8dnnx" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273793    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="36af1207-9951-4160-bb36-6dd100a05abb" volumeName="kubernetes.io/projected/36af1207-9951-4160-bb36-6dd100a05abb-kube-api-access-kcpp8" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273806    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="39429a51-69a0-4bce-9029-773d5aad2a87" volumeName="kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273829    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50f36ad6-52cd-4239-94eb-22b8f3de2a26" volumeName="kubernetes.io/configmap/50f36ad6-52cd-4239-94eb-22b8f3de2a26-iptables-alerter-script" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273857    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50f36ad6-52cd-4239-94eb-22b8f3de2a26" volumeName="kubernetes.io/projected/50f36ad6-52cd-4239-94eb-22b8f3de2a26-kube-api-access-grkj6" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273869    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae2e6da6-1e6d-4b54-8136-359490273f9c" volumeName="kubernetes.io/projected/ae2e6da6-1e6d-4b54-8136-359490273f9c-kube-api-access-wz7p8" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273884    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0c7e7cd-abcd-429b-b556-947ac5d9402a" volumeName="kubernetes.io/configmap/e0c7e7cd-abcd-429b-b556-947ac5d9402a-cni-binary-copy" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273897    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="39640be1-9711-4cef-a9a2-430ea40c7c18" volumeName="kubernetes.io/secret/39640be1-9711-4cef-a9a2-430ea40c7c18-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273910    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7586e6a7-b118-4cef-9c67-56e5a5d4eb04" volumeName="kubernetes.io/empty-dir/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-tuned" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273921    4420 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1f6fbb1-af6e-43e4-a671-68a6c3691463" volumeName="kubernetes.io/empty-dir/e1f6fbb1-af6e-43e4-a671-68a6c3691463-tmp-dir" seLinuxMountContext=""
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273933    4420 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.273941    4420 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.274356    4420 factory.go:55] Registering systemd factory
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.274375    4420 factory.go:223] Registration of the systemd container factory successfully
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.275204    4420 factory.go:153] Registering CRI-O factory
Nov 04 03:17:10.275773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.275233    4420 factory.go:223] Registration of the crio container factory successfully
Nov 04 03:17:10.277728 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.277704    4420 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 03:17:10.277796 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.277760    4420 factory.go:103] Registering Raw factory
Nov 04 03:17:10.277796 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.277782    4420 manager.go:1199] Started watching for new ooms in manager
Nov 04 03:17:10.278495 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.278481    4420 manager.go:319] Starting recovery of all containers
Nov 04 03:17:10.297864 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.297807    4420 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 03:17:10.299264 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.299243    4420 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 03:17:10.299365 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.299272    4420 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 03:17:10.299365 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.299294    4420 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 03:17:10.299365 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:10.299338    4420 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 03:17:10.320625 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.320588    4420 generic.go:356] "Generic (PLEG): container finished" podID="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" containerID="621ded60eed8678e8b1f9a99fa5f2bbe6667a8789b69b90d5e14fe585068b5e5" exitCode=0
Nov 04 03:17:10.320625 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.320621    4420 generic.go:356] "Generic (PLEG): container finished" podID="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" containerID="82b2d1cf56f50ffbe09754622d8198251be6e521c938c5fc251469be1c039441" exitCode=0
Nov 04 03:17:10.320625 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.320637    4420 generic.go:356] "Generic (PLEG): container finished" podID="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" containerID="860591cc6810b7f28e163829b6db691932a35e0698eed0699c30822dff3960d1" exitCode=0
Nov 04 03:17:10.320900 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.320645    4420 generic.go:356] "Generic (PLEG): container finished" podID="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" containerID="d93f00c373a9e1a7a49332d5b0c3ef8dd9ecd97e69b160fdd56a013d07ccd8d9" exitCode=0
Nov 04 03:17:10.320900 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.320654    4420 generic.go:356] "Generic (PLEG): container finished" podID="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" containerID="126f6eb7d0a6858661b9fffbecef6e49b12c4281d35a25323d4b625225dbac32" exitCode=0
Nov 04 03:17:10.320900 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.320662    4420 generic.go:356] "Generic (PLEG): container finished" podID="b4f10f3e-8fd4-4e4e-acc1-3c5622796acd" containerID="d6221af860fc44d21107155b2c0f6b3b5bc75cfe9ae5cbddb9bd475f87b145cb" exitCode=0
Nov 04 03:17:10.331205 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.331167    4420 generic.go:356] "Generic (PLEG): container finished" podID="4d7792c1def8a40547a57207fd144414" containerID="a5112290b882e1bb49f444afa2a5ef8a39966795db3695790e961d246561b9a2" exitCode=0
Nov 04 03:17:10.343159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.343136    4420 manager.go:324] Recovery completed
Nov 04 03:17:10.354585 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.354572    4420 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:10.355630 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.355611    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:10.355702 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.355641    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:10.355702 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.355651    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:10.356898 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.356883    4420 cpu_manager.go:226] "Starting" policy="none"
Nov 04 03:17:10.356898 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.356895    4420 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:10.356969 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.356896    4420 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 03:17:10.356969 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.356965    4420 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 03:17:10.357114 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357099    4420 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 03:17:10.357146 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357122    4420 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 03:17:10.357146 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357140    4420 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 03:17:10.357146 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357145    4420 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 03:17:10.357248 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357151    4420 policy_none.go:50] "Start"
Nov 04 03:17:10.357248 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357162    4420 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 03:17:10.357248 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357171    4420 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 03:17:10.357371 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357305    4420 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 03:17:10.357371 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357312    4420 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 03:17:10.357371 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357318    4420 policy_none.go:44] "Start"
Nov 04 03:17:10.357788 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357774    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:10.357901 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357798    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:10.357901 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.357808    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:10.362470 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.362436    4420 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 03:17:10.365085 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.365071    4420 manager.go:341] "Starting Device Plugin manager"
Nov 04 03:17:10.365183 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:10.365100    4420 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 03:17:10.365183 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.365112    4420 server.go:87] "Starting device plugin registration server"
Nov 04 03:17:10.365409 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.365395    4420 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 03:17:10.365472 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.365411    4420 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 03:17:10.365528 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.365490    4420 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 03:17:10.365626 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.365593    4420 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 03:17:10.365696 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.365634    4420 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 03:17:10.365696 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.365641    4420 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 03:17:10.365794 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.365697    4420 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T03:17:10.365623368Z","UUID":"6624834a-01b1-4367-abfe-d150d61e955b","Handler":null,"Name":"","Endpoint":""}
Nov 04 03:17:10.367172 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:10.367150    4420 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 03:17:10.367253 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:10.367191    4420 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-201.us-west-2.compute.internal\" not found"
Nov 04 03:17:10.367253 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.367158    4420 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 03:17:10.367253 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.367216    4420 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 03:17:10.400353 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.399895    4420 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"]
Nov 04 03:17:10.400441 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.400391    4420 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:10.401212 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.401196    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:10.401273 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.401226    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:10.401273 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.401239    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:10.401485 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.401475    4420 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:10.401609 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.401594    4420 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:10.402204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.402189    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:10.402262 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.402216    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:10.402262 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.402225    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:10.402262 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.402231    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:10.402262 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.402243    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:10.402262 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.402256    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:10.402414 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:10.402387    4420 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found" node="ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.402582 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.402550    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal" event={"ID":"4d7792c1def8a40547a57207fd144414","Type":"ContainerStarted","Data":"ac555a88ec118d69ff1a06ad35df562db17bd05e6c25f8d9ff71ce49269c5e7d"}
Nov 04 03:17:10.402617 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.402595    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal" event={"ID":"4d7792c1def8a40547a57207fd144414","Type":"ContainerDied","Data":"a5112290b882e1bb49f444afa2a5ef8a39966795db3695790e961d246561b9a2"}
Nov 04 03:17:10.402647 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.402616    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal" event={"ID":"4d7792c1def8a40547a57207fd144414","Type":"ContainerStarted","Data":"8dc619aefb0904ee71ece245244cf08e87f4e4a160db525930f37e2107a07489"}
Nov 04 03:17:10.402647 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.402627    4420 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:10.402647 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.402636    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal" event={"ID":"72ddb4465d8ebfdcb6c7be795263771f","Type":"ContainerStarted","Data":"837fb68afd690ffdac3a41605531db338b6a09ed98d2eb67467be85386374a18"}
Nov 04 03:17:10.402793 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.402655    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal" event={"ID":"72ddb4465d8ebfdcb6c7be795263771f","Type":"ContainerStarted","Data":"6fb613bc7c958935a8d23066b5bc51099623fab47f72c4dd27825c6579aba8b8"}
Nov 04 03:17:10.403275 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.403263    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:10.403328 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.403284    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:10.403328 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.403293    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:10.403407 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:10.403396    4420 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-201.us-west-2.compute.internal\" not found" node="ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.458456 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.458432    4420 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:17:10.458585 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.458572    4420 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 03:17:10.459316 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.459302    4420 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:17:10.459439 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.459424    4420 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:17:10.465663 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.465649    4420 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:17:10.466513 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.466498    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:17:10.466581 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.466522    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:17:10.466581 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.466533    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:17:10.466581 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.466561    4420 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.473746 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.473733    4420 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.506143 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.506123    4420 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:17:10.550767 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.550747    4420 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:17:10.550884 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:10.550785    4420 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.550884 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.550797    4420 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.558815 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.558798    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d7792c1def8a40547a57207fd144414-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"4d7792c1def8a40547a57207fd144414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.558882 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.558824    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d7792c1def8a40547a57207fd144414-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"4d7792c1def8a40547a57207fd144414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.558882 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.558863    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72ddb4465d8ebfdcb6c7be795263771f-config\") pod \"kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"72ddb4465d8ebfdcb6c7be795263771f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.568822 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.568808    4420 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.568889 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.568879    4420 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.569385 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.569369    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeNotReady"
Nov 04 03:17:10.569442 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.569397    4420 setters.go:546] "Node became not ready" node="ip-10-0-133-201.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-04T03:17:10Z","lastTransitionTime":"2026-11-04T03:17:10Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 04 03:17:10.612574 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.612544    4420 nodeinfomanager.go:471] existing CSINode "ip-10-0-133-201.us-west-2.compute.internal" is owned by different node (oldNodeID="766a9ab5-ecab-428a-9e4d-0c0435a2497e", newNodeID=""), cleaning up...
Nov 04 03:17:10.635628 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.635602    4420 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:17:10.635730 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:10.635648    4420 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.659579 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.659526    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d7792c1def8a40547a57207fd144414-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"4d7792c1def8a40547a57207fd144414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.659579 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.659554    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72ddb4465d8ebfdcb6c7be795263771f-config\") pod \"kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"72ddb4465d8ebfdcb6c7be795263771f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.659579 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.659571    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d7792c1def8a40547a57207fd144414-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"4d7792c1def8a40547a57207fd144414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.659698 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.659610    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d7792c1def8a40547a57207fd144414-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"4d7792c1def8a40547a57207fd144414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.659698 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.659638    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72ddb4465d8ebfdcb6c7be795263771f-config\") pod \"kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"72ddb4465d8ebfdcb6c7be795263771f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:10.659698 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:10.659638    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d7792c1def8a40547a57207fd144414-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" (UID: \"4d7792c1def8a40547a57207fd144414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:11.259346 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.259315    4420 apiserver.go:52] "Watching apiserver"
Nov 04 03:17:11.336655 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.336625    4420 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:11.336815 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.336790    4420 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:11.355829 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.355804    4420 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 03:17:11.356210 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.356190    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-rz4mh","openshift-network-operator/iptables-alerter-n6s5j","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px","openshift-ingress-canary/ingress-canary-2kdk2","openshift-cluster-node-tuning-operator/tuned-l74dq","openshift-multus/multus-additional-cni-plugins-s2tvj","kube-system/global-pull-secret-syncer-tmrtb","openshift-dns/dns-default-ckjtr","openshift-ovn-kubernetes/ovnkube-node-vg574","kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal","openshift-insights/insights-runtime-extractor-vtl99","openshift-multus/multus-qsm2l","openshift-network-diagnostics/network-check-target-5zxn5","kube-system/konnectivity-agent-6x62l","openshift-image-registry/node-ca-8r4z6","openshift-kube-storage-version-migrator/migrator-66df5d496b-llrgs","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal","openshift-dns/node-resolver-gccvl"]
Nov 04 03:17:11.359920 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.359903    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:11.361050 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.361033    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:11.361926 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.361905    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:11.373523 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.373506    4420 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 03:17:11.393928 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.393903    4420 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:17:11.394005 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:11.393952    4420 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:11.394515 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.394504    4420 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:17:11.394562 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:11.394532    4420 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-201.us-west-2.compute.internal"
Nov 04 03:17:11.414336 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414301    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 03:17:11.414336 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414302    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:11.414336 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414313    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 03:17:11.414336 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414318    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 03:17:11.414575 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414559    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 03:17:11.414701 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414578    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:11.414701 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414682    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 03:17:11.414856 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414823    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 03:17:11.414856 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414824    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 03:17:11.414981 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414855    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 03:17:11.414981 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414889    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 03:17:11.414981 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414888    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 03:17:11.414981 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.414924    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 03:17:11.415191 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415176    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 03:17:11.415264 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415197    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 03:17:11.415264 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415228    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 03:17:11.415264 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415229    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 03:17:11.415384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415291    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 03:17:11.415384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415329    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 03:17:11.415384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415351    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4vrb5\""
Nov 04 03:17:11.415384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415376    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 03:17:11.415384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415376    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-56wj9\""
Nov 04 03:17:11.415586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415393    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x6xxb\""
Nov 04 03:17:11.415586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415351    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zsv6g\""
Nov 04 03:17:11.415586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415419    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z2nst\""
Nov 04 03:17:11.415586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415420    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h66cx\""
Nov 04 03:17:11.415586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415438    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 03:17:11.415586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415453    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 03:17:11.415586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415478    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 03:17:11.415586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415485    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fbcnm\""
Nov 04 03:17:11.415586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415479    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 03:17:11.415586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415540    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 03:17:11.415991 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415595    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-vpndz\""
Nov 04 03:17:11.415991 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415638    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 03:17:11.415991 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.415717    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9cxzk\""
Nov 04 03:17:11.416379 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.416358    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 03:17:11.463704 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.463668    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-slash\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.463704 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.463700    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36af1207-9951-4160-bb36-6dd100a05abb-tmp-dir\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:11.463922 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.463719    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cni-binary-copy\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.463922 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.463735    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.463922 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.463752    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-systemd-units\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.463922 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.463817    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-cni-netd\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.463922 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.463878    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36af1207-9951-4160-bb36-6dd100a05abb-tmp-dir\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:11.463922 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.463889    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39640be1-9711-4cef-a9a2-430ea40c7c18-ovn-node-metrics-cert\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.463929    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.463958    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.463983    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50f36ad6-52cd-4239-94eb-22b8f3de2a26-host-slash\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464005    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-netns\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464028    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1f6fbb1-af6e-43e4-a671-68a6c3691463-hosts-file\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464073    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl5th\" (UniqueName: \"kubernetes.io/projected/e1685aa8-41b4-4899-ad2d-6e03e2969d72-kube-api-access-bl5th\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464094    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39640be1-9711-4cef-a9a2-430ea40c7c18-ovn-node-metrics-cert\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464115    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-system-cni-dir\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464147    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grkj6\" (UniqueName: \"kubernetes.io/projected/50f36ad6-52cd-4239-94eb-22b8f3de2a26-kube-api-access-grkj6\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464178    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-hostroot\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464197    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cni-binary-copy\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464207    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/39429a51-69a0-4bce-9029-773d5aad2a87-original-pull-secret\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:17:11.464217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464204    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-sys\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.464635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464242    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-lib-modules\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.464635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464273    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0599c4e4-3db5-42b2-bd74-2c874ab74383-host\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:17:11.464635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464304    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.464635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464336    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/39429a51-69a0-4bce-9029-773d5aad2a87-dbus\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:17:11.464635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464361    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-system-cni-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.464635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464391    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-cnibin\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.464635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464411    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-k8s-cni-cncf-io\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.464635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464438    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-daemon-config\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.464635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464493    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7nf4\" (UniqueName: \"kubernetes.io/projected/99ab9861-53bc-4c46-a53b-f0a0b1a2c53d-kube-api-access-w7nf4\") pod \"migrator-66df5d496b-llrgs\" (UID: \"99ab9861-53bc-4c46-a53b-f0a0b1a2c53d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-llrgs"
Nov 04 03:17:11.464635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464531    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-run-netns\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.464635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464572    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cnibin\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.464635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464617    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5st2r\" (UniqueName: \"kubernetes.io/projected/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-kube-api-access-5st2r\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464648    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0c7e7cd-abcd-429b-b556-947ac5d9402a-cni-binary-copy\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464674    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-cni-multus\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464762    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-etc-kubernetes\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464786    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22q58\" (UniqueName: \"kubernetes.io/projected/e0c7e7cd-abcd-429b-b556-947ac5d9402a-kube-api-access-22q58\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464807    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464855    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-ovnkube-config\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464922    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0c7e7cd-abcd-429b-b556-947ac5d9402a-cni-binary-copy\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464943    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.464976    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-conf-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465066    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-tuned\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465092    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-tmp\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465117    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-var-lib-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465153    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-tuned\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465159    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-socket-dir-parent\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465168    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-tmp\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465170    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-ovnkube-config\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465200    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-kubelet\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.465232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465233    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-systemd\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465264    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsm6h\" (UniqueName: \"kubernetes.io/projected/39640be1-9711-4cef-a9a2-430ea40c7c18-kube-api-access-tsm6h\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465312    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/36af1207-9951-4160-bb36-6dd100a05abb-config-volume\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465337    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-host\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465361    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-kubelet\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465439    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3aef6eaf-2dde-4706-bcf9-77bf9c447028-agent-certs\") pod \"konnectivity-agent-6x62l\" (UID: \"3aef6eaf-2dde-4706-bcf9-77bf9c447028\") " pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465457    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8830f068-f4c8-492e-8234-4dbf4456f3e5-crio-socket\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465474    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqp7t\" (UniqueName: \"kubernetes.io/projected/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-api-access-sqp7t\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465490    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysctl-d\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465506    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-ovnkube-script-lib\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465540    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-cni-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465573    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lswx\" (UniqueName: \"kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx\") pod \"network-check-target-5zxn5\" (UID: \"7566445c-b5ac-424d-aa38-a2c5b2e0de21\") " pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465593    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-etc-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465639    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-device-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465657    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-multus-certs\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465671    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysctl-conf\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465680    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-ovnkube-script-lib\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465694    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzzhc\" (UniqueName: \"kubernetes.io/projected/0599c4e4-3db5-42b2-bd74-2c874ab74383-kube-api-access-rzzhc\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465702    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3aef6eaf-2dde-4706-bcf9-77bf9c447028-agent-certs\") pod \"konnectivity-agent-6x62l\" (UID: \"3aef6eaf-2dde-4706-bcf9-77bf9c447028\") " pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465756    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-log-socket\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465809    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcpp8\" (UniqueName: \"kubernetes.io/projected/36af1207-9951-4160-bb36-6dd100a05abb-kube-api-access-kcpp8\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465837    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465879    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1f6fbb1-af6e-43e4-a671-68a6c3691463-tmp-dir\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465905    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-var-lib-kubelet\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465932    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0599c4e4-3db5-42b2-bd74-2c874ab74383-serviceca\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:17:11.465982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.465977    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-node-log\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466003    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1f6fbb1-af6e-43e4-a671-68a6c3691463-tmp-dir\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466048    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-run-ovn-kubernetes\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466060    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466085    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-cni-bin\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466106    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-etc-selinux\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466132    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjqlf\" (UniqueName: \"kubernetes.io/projected/6cec1f32-6464-4117-b098-2ed5b3c9cf64-kube-api-access-vjqlf\") pod \"ingress-canary-2kdk2\" (UID: \"6cec1f32-6464-4117-b098-2ed5b3c9cf64\") " pod="openshift-ingress-canary/ingress-canary-2kdk2"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466153    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0599c4e4-3db5-42b2-bd74-2c874ab74383-serviceca\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466184    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8830f068-f4c8-492e-8234-4dbf4456f3e5-data-volume\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466202    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-cni-bin\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466218    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-modprobe-d\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466221    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8830f068-f4c8-492e-8234-4dbf4456f3e5-data-volume\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466246    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/39429a51-69a0-4bce-9029-773d5aad2a87-kubelet-config\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466266    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466314    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466335    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cec1f32-6464-4117-b098-2ed5b3c9cf64-cert\") pod \"ingress-canary-2kdk2\" (UID: \"6cec1f32-6464-4117-b098-2ed5b3c9cf64\") " pod="openshift-ingress-canary/ingress-canary-2kdk2"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466350    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dnnx\" (UniqueName: \"kubernetes.io/projected/e1f6fbb1-af6e-43e4-a671-68a6c3691463-kube-api-access-8dnnx\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466366    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-kubernetes\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466381    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-run\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466400    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-ovn\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466441    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466473    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-env-overrides\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466508    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-registration-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466525    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-sys-fs\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466536    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466542    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz7p8\" (UniqueName: \"kubernetes.io/projected/ae2e6da6-1e6d-4b54-8136-359490273f9c-kube-api-access-wz7p8\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466570    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3aef6eaf-2dde-4706-bcf9-77bf9c447028-konnectivity-ca\") pod \"konnectivity-agent-6x62l\" (UID: \"3aef6eaf-2dde-4706-bcf9-77bf9c447028\") " pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466644    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39640be1-9711-4cef-a9a2-430ea40c7c18-env-overrides\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466659    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysconfig\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466678    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3aef6eaf-2dde-4706-bcf9-77bf9c447028-konnectivity-ca\") pod \"konnectivity-agent-6x62l\" (UID: \"3aef6eaf-2dde-4706-bcf9-77bf9c447028\") " pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466687    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-socket-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466714    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/50f36ad6-52cd-4239-94eb-22b8f3de2a26-iptables-alerter-script\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466782    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466812    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-systemd\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.466833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466835    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9499\" (UniqueName: \"kubernetes.io/projected/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-kube-api-access-s9499\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.467873 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.466892    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/50f36ad6-52cd-4239-94eb-22b8f3de2a26-iptables-alerter-script\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:17:11.514294 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.514229    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 03:17:11.514396 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.514380    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 03:17:11.514432 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.514387    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 03:17:11.514432 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.514403    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 03:17:11.514432 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.514411    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 03:17:11.514553 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.514542    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 03:17:11.514592 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.514582    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 03:17:11.514640 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.514596    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 03:17:11.514640 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.514608    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 03:17:11.514717 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.514649    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 03:17:11.514790 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.514776    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 03:17:11.514836 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.514819    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 03:17:11.515062 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.515049    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5zf5b\""
Nov 04 03:17:11.515101 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.515081    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 03:17:11.515101 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.515073    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 03:17:11.515169 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.515156    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jw2mq\""
Nov 04 03:17:11.515214 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.515174    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vjndb\""
Nov 04 03:17:11.515214 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.515207    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-mnqmt\""
Nov 04 03:17:11.515288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.515212    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-569bn\""
Nov 04 03:17:11.515288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.515232    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-mhdb8\""
Nov 04 03:17:11.515288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.515223    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xwzp4\""
Nov 04 03:17:11.515288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.515212    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 03:17:11.516513 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.516499    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/36af1207-9951-4160-bb36-6dd100a05abb-config-volume\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:11.516570 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.516558    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cec1f32-6464-4117-b098-2ed5b3c9cf64-cert\") pod \"ingress-canary-2kdk2\" (UID: \"6cec1f32-6464-4117-b098-2ed5b3c9cf64\") " pod="openshift-ingress-canary/ingress-canary-2kdk2"
Nov 04 03:17:11.520067 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.520052    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7nf4\" (UniqueName: \"kubernetes.io/projected/99ab9861-53bc-4c46-a53b-f0a0b1a2c53d-kube-api-access-w7nf4\") pod \"migrator-66df5d496b-llrgs\" (UID: \"99ab9861-53bc-4c46-a53b-f0a0b1a2c53d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-llrgs"
Nov 04 03:17:11.520650 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.520631    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22q58\" (UniqueName: \"kubernetes.io/projected/e0c7e7cd-abcd-429b-b556-947ac5d9402a-kube-api-access-22q58\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.520734 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.520638    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5st2r\" (UniqueName: \"kubernetes.io/projected/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-kube-api-access-5st2r\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.520798 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.520745    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsm6h\" (UniqueName: \"kubernetes.io/projected/39640be1-9711-4cef-a9a2-430ea40c7c18-kube-api-access-tsm6h\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.520798 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.520765    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl5th\" (UniqueName: \"kubernetes.io/projected/e1685aa8-41b4-4899-ad2d-6e03e2969d72-kube-api-access-bl5th\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.524633 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.524614    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-daemon-config\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.524988 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.524973    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grkj6\" (UniqueName: \"kubernetes.io/projected/50f36ad6-52cd-4239-94eb-22b8f3de2a26-kube-api-access-grkj6\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:17:11.526330 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.526307    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzzhc\" (UniqueName: \"kubernetes.io/projected/0599c4e4-3db5-42b2-bd74-2c874ab74383-kube-api-access-rzzhc\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:17:11.526330 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.526317    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqp7t\" (UniqueName: \"kubernetes.io/projected/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-api-access-sqp7t\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:11.526468 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.526413    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lswx\" (UniqueName: \"kubernetes.io/projected/7566445c-b5ac-424d-aa38-a2c5b2e0de21-kube-api-access-5lswx\") pod \"network-check-target-5zxn5\" (UID: \"7566445c-b5ac-424d-aa38-a2c5b2e0de21\") " pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:17:11.526748 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.526730    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dnnx\" (UniqueName: \"kubernetes.io/projected/e1f6fbb1-af6e-43e4-a671-68a6c3691463-kube-api-access-8dnnx\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:17:11.527105 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.527087    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9499\" (UniqueName: \"kubernetes.io/projected/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-kube-api-access-s9499\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.527194 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.527180    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz7p8\" (UniqueName: \"kubernetes.io/projected/ae2e6da6-1e6d-4b54-8136-359490273f9c-kube-api-access-wz7p8\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:11.527233 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.527195    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcpp8\" (UniqueName: \"kubernetes.io/projected/36af1207-9951-4160-bb36-6dd100a05abb-kube-api-access-kcpp8\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:11.527528 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.527513    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjqlf\" (UniqueName: \"kubernetes.io/projected/6cec1f32-6464-4117-b098-2ed5b3c9cf64-kube-api-access-vjqlf\") pod \"ingress-canary-2kdk2\" (UID: \"6cec1f32-6464-4117-b098-2ed5b3c9cf64\") " pod="openshift-ingress-canary/ingress-canary-2kdk2"
Nov 04 03:17:11.567392 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567372    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-host\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567439 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567397    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-kubelet\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567439 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567414    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8830f068-f4c8-492e-8234-4dbf4456f3e5-crio-socket\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:11.567439 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567432    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysctl-d\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567451    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-cni-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.567571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567466    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-etc-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567482    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-device-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.567571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567493    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8830f068-f4c8-492e-8234-4dbf4456f3e5-crio-socket\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:11.567571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567498    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-multus-certs\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.567571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567516    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-host\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567465    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-kubelet\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567532    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-cni-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.567571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567541    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-multus-certs\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.567571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567520    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-etc-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567566    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysctl-conf\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567582    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysctl-d\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567568    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-device-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567609    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-log-socket\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567634    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-log-socket\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567642    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-var-lib-kubelet\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567646    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysctl-conf\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567673    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-var-lib-kubelet\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567665    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-node-log\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567702    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-run-ovn-kubernetes\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567717    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-cni-bin\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567732    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-etc-selinux\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567755    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-cni-bin\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567764    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-run-ovn-kubernetes\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567780    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-modprobe-d\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567791    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-cni-bin\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567793    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-node-log\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567804    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/39429a51-69a0-4bce-9029-773d5aad2a87-kubelet-config\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567820    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-cni-bin\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567827    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567870    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-kubernetes\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567904    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-run\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567920    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-kubernetes\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567923    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-modprobe-d\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567929    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/39429a51-69a0-4bce-9029-773d5aad2a87-kubelet-config\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567805    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-etc-selinux\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567927    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-ovn\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.567923 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:11.567920    4420 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567946    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-run\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.567971    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-ovn\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568015    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:11.568056    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs podName:ae2e6da6-1e6d-4b54-8136-359490273f9c nodeName:}" failed. No retries permitted until 2026-11-04 03:17:12.068041397 +0000 UTC m=+1.882832652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs") pod "network-metrics-daemon-rz4mh" (UID: "ae2e6da6-1e6d-4b54-8136-359490273f9c") : secret "metrics-daemon-secret" not found
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568083    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-registration-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568086    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568100    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-sys-fs\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568116    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysconfig\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568124    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-registration-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568129    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-socket-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568144    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568164    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-systemd\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568181    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-slash\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568163    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-sys-fs\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:11.568199    4420 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568203    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-socket-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568204    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568214    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-systemd\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568188    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-etc-sysconfig\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568255    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-slash\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568269    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568277    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-systemd-units\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:11.568352    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:12.06830649 +0000 UTC m=+1.883097749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568370    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-cni-netd\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568371    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-systemd-units\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568413    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-cni-netd\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568444    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568476    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50f36ad6-52cd-4239-94eb-22b8f3de2a26-host-slash\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:11.568517    4420 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:11.568566    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls podName:36af1207-9951-4160-bb36-6dd100a05abb nodeName:}" failed. No retries permitted until 2026-11-04 03:17:12.06855249 +0000 UTC m=+1.883343759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls") pod "dns-default-ckjtr" (UID: "36af1207-9951-4160-bb36-6dd100a05abb") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568583    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-netns\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568606    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1f6fbb1-af6e-43e4-a671-68a6c3691463-hosts-file\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568622    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-system-cni-dir\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568582    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50f36ad6-52cd-4239-94eb-22b8f3de2a26-host-slash\") pod \"iptables-alerter-n6s5j\" (UID: \"50f36ad6-52cd-4239-94eb-22b8f3de2a26\") " pod="openshift-network-operator/iptables-alerter-n6s5j"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568640    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-hostroot\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568644    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-netns\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568674    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-system-cni-dir\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568682    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-hostroot\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568682    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1f6fbb1-af6e-43e4-a671-68a6c3691463-hosts-file\") pod \"node-resolver-gccvl\" (UID: \"e1f6fbb1-af6e-43e4-a671-68a6c3691463\") " pod="openshift-dns/node-resolver-gccvl"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568709    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-sys\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.568703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568726    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-lib-modules\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568741    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0599c4e4-3db5-42b2-bd74-2c874ab74383-host\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568767    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0599c4e4-3db5-42b2-bd74-2c874ab74383-host\") pod \"node-ca-8r4z6\" (UID: \"0599c4e4-3db5-42b2-bd74-2c874ab74383\") " pod="openshift-image-registry/node-ca-8r4z6"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568770    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-sys\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568784    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568806    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/39429a51-69a0-4bce-9029-773d5aad2a87-dbus\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568817    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e1685aa8-41b4-4899-ad2d-6e03e2969d72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r47px\" (UID: \"e1685aa8-41b4-4899-ad2d-6e03e2969d72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r47px"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568821    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-system-cni-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568820    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7586e6a7-b118-4cef-9c67-56e5a5d4eb04-lib-modules\") pod \"tuned-l74dq\" (UID: \"7586e6a7-b118-4cef-9c67-56e5a5d4eb04\") " pod="openshift-cluster-node-tuning-operator/tuned-l74dq"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568854    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-cnibin\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568870    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-k8s-cni-cncf-io\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568885    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-run-netns\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568889    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/39429a51-69a0-4bce-9029-773d5aad2a87-dbus\") pod \"global-pull-secret-syncer-tmrtb\" (UID: \"39429a51-69a0-4bce-9029-773d5aad2a87\") " pod="kube-system/global-pull-secret-syncer-tmrtb"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568900    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cnibin\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568920    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-system-cni-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568927    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-cni-multus\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568948    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-cni-multus\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568948    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-host-run-netns\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568947    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-run-k8s-cni-cncf-io\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568953    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-etc-kubernetes\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568974    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-cnibin\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568980    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-etc-kubernetes\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568987    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.569006    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.568987    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4f10f3e-8fd4-4e4e-acc1-3c5622796acd-cnibin\") pod \"multus-additional-cni-plugins-s2tvj\" (UID: \"b4f10f3e-8fd4-4e4e-acc1-3c5622796acd\") " pod="openshift-multus/multus-additional-cni-plugins-s2tvj"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.569038    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.569069    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-conf-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.569097    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-var-lib-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.569122    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-conf-dir\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.569124    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-var-lib-openvswitch\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:11.569125    4420 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.569122    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-socket-dir-parent\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.569162    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-kubelet\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:11.569172    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:12.0691621 +0000 UTC m=+1.883953357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.569184    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-host-var-lib-kubelet\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.569210    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-systemd\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.569215    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0c7e7cd-abcd-429b-b556-947ac5d9402a-multus-socket-dir-parent\") pod \"multus-qsm2l\" (UID: \"e0c7e7cd-abcd-429b-b556-947ac5d9402a\") " pod="openshift-multus/multus-qsm2l"
Nov 04 03:17:11.569858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.569254    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39640be1-9711-4cef-a9a2-430ea40c7c18-run-systemd\") pod \"ovnkube-node-vg574\" (UID: \"39640be1-9711-4cef-a9a2-430ea40c7c18\") " pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:11.981759 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:11.981680    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:12.072793 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:12.072755    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:12.072952 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:12.072801    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:12.072952 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:12.072821    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:12.072952 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:12.072866    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:12.072952 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:12.072934    4420 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:12.072952 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:12.072948    4420 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:12.073145 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:12.072961    4420 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:12.073145 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:12.072934    4420 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 03:17:12.073145 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:12.072997    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:13.07298435 +0000 UTC m=+2.887775605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:12.073145 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:12.073016    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls podName:36af1207-9951-4160-bb36-6dd100a05abb nodeName:}" failed. No retries permitted until 2026-11-04 03:17:13.073003962 +0000 UTC m=+2.887795216 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls") pod "dns-default-ckjtr" (UID: "36af1207-9951-4160-bb36-6dd100a05abb") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:12.073145 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:12.073031    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs podName:ae2e6da6-1e6d-4b54-8136-359490273f9c nodeName:}" failed. No retries permitted until 2026-11-04 03:17:13.073023899 +0000 UTC m=+2.887815157 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs") pod "network-metrics-daemon-rz4mh" (UID: "ae2e6da6-1e6d-4b54-8136-359490273f9c") : secret "metrics-daemon-secret" not found
Nov 04 03:17:12.073145 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:12.073049    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:13.073043766 +0000 UTC m=+2.887835020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:12.128488 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:12.128452    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:13.077633 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:13.077592    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:13.077633 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:13.077638    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:13.078130 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:13.077664    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:13.078130 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:13.077688    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:13.078130 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:13.077757    4420 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:13.078130 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:13.077775    4420 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:13.078130 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:13.077779    4420 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 03:17:13.078130 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:13.077824    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:15.077811411 +0000 UTC m=+4.892602666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:13.078130 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:13.077835    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:15.077830025 +0000 UTC m=+4.892621279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:13.078130 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:13.077783    4420 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 03:17:13.078130 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:13.077890    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls podName:36af1207-9951-4160-bb36-6dd100a05abb nodeName:}" failed. No retries permitted until 2026-11-04 03:17:15.077875245 +0000 UTC m=+4.892666525 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls") pod "dns-default-ckjtr" (UID: "36af1207-9951-4160-bb36-6dd100a05abb") : secret "dns-default-metrics-tls" not found
Nov 04 03:17:13.078130 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:13.077922    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs podName:ae2e6da6-1e6d-4b54-8136-359490273f9c nodeName:}" failed. No retries permitted until 2026-11-04 03:17:15.07790475 +0000 UTC m=+4.892696022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs") pod "network-metrics-daemon-rz4mh" (UID: "ae2e6da6-1e6d-4b54-8136-359490273f9c") : secret "metrics-daemon-secret" not found
Nov 04 03:17:13.098681 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:13.098647    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:13.398406 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:13.398326    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:13.541606 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:13.541579    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gccvl_e1f6fbb1-af6e-43e4-a671-68a6c3691463/dns-node-resolver/0.log"
Nov 04 03:17:13.640915 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:13.640885    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:13.788373 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:13.788338    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:14.016711 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:14.016683    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:14.090604 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:14.090526    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:14.298916 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:14.298876    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:14.340929 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:14.340862    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8r4z6_0599c4e4-3db5-42b2-bd74-2c874ab74383/node-ca/0.log"
Nov 04 03:17:14.495270 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:14.495236    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:15.092275 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.092245    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:15.092680 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.092286    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:15.092680 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.092308    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:15.092680 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.092343    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:15.092680 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:15.092379    4420 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:15.092680 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:15.092428    4420 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:15.092680 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:15.092443    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:19.092429095 +0000 UTC m=+8.907220369 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:15.092680 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:15.092476    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:19.092462628 +0000 UTC m=+8.907253883 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:15.092680 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.092648    4420 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 03:17:15.094655 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.094633    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae2e6da6-1e6d-4b54-8136-359490273f9c-metrics-certs\") pod \"network-metrics-daemon-rz4mh\" (UID: \"ae2e6da6-1e6d-4b54-8136-359490273f9c\") " pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:15.094780 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.094633    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36af1207-9951-4160-bb36-6dd100a05abb-metrics-tls\") pod \"dns-default-ckjtr\" (UID: \"36af1207-9951-4160-bb36-6dd100a05abb\") " pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:15.140583 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.140558    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2kdk2_6cec1f32-6464-4117-b098-2ed5b3c9cf64/serve-healthcheck-canary/0.log"
Nov 04 03:17:15.260822 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.260791    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rz4mh"
Nov 04 03:17:15.261935 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.261917    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:15.291335 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.291251    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:15.491518 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.491489    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:17:15.648109 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.648078    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rz4mh"]
Nov 04 03:17:15.676611 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:15.676582    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2e6da6_1e6d_4b54_8136_359490273f9c.slice/crio-a2f96c1b2a07edea31092a0f8458c3a55cc0399c2f1bf85a6d44a1a51db37a3e WatchSource:0}: Error finding container a2f96c1b2a07edea31092a0f8458c3a55cc0399c2f1bf85a6d44a1a51db37a3e: Status 404 returned error can't find the container with id a2f96c1b2a07edea31092a0f8458c3a55cc0399c2f1bf85a6d44a1a51db37a3e
Nov 04 03:17:15.678551 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.678537    4420 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:17:15.686505 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.686484    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ckjtr"]
Nov 04 03:17:15.689377 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:15.689354    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36af1207_9951_4160_bb36_6dd100a05abb.slice/crio-2a2679849f605a365b1d742a8a3acd354a3b9f310a3a7c68c60282b2f5e0fa3d WatchSource:0}: Error finding container 2a2679849f605a365b1d742a8a3acd354a3b9f310a3a7c68c60282b2f5e0fa3d: Status 404 returned error can't find the container with id 2a2679849f605a365b1d742a8a3acd354a3b9f310a3a7c68c60282b2f5e0fa3d
Nov 04 03:17:15.697975 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.697949    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:15.895725 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.895647    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:15.920270 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.920246    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:15.920440 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.920426    4420 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:17:15.941487 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:15.941463    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-llrgs_99ab9861-53bc-4c46-a53b-f0a0b1a2c53d/migrator/0.log"
Nov 04 03:17:16.091486 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:16.091447    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:16.347362 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:16.347328    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ckjtr" event={"ID":"36af1207-9951-4160-bb36-6dd100a05abb","Type":"ContainerStarted","Data":"2a2679849f605a365b1d742a8a3acd354a3b9f310a3a7c68c60282b2f5e0fa3d"}
Nov 04 03:17:16.348478 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:16.348444    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rz4mh" event={"ID":"ae2e6da6-1e6d-4b54-8136-359490273f9c","Type":"ContainerStarted","Data":"a2f96c1b2a07edea31092a0f8458c3a55cc0399c2f1bf85a6d44a1a51db37a3e"}
Nov 04 03:17:16.498028 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:16.497991    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:17.322655 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:17.322615    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:17.692901 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:17.692814    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:17.894920 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:17.894823    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:18.095169 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:18.095130    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:18.355757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:18.355719    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rz4mh" event={"ID":"ae2e6da6-1e6d-4b54-8136-359490273f9c","Type":"ContainerStarted","Data":"387ede9cca4dbb4045c563a847f918255da5cf5f471a0ac3607bee7f90f51749"}
Nov 04 03:17:18.355970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:18.355775    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rz4mh" event={"ID":"ae2e6da6-1e6d-4b54-8136-359490273f9c","Type":"ContainerStarted","Data":"17e4238baf79b2e7120196f46ce2956a9616943d9a3e8707cba8c5a4d20c7a55"}
Nov 04 03:17:18.496441 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:18.496408    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:19.119381 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:19.119345    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:19.119756 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:19.119392    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:19.119756 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:19.119465    4420 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 03:17:19.119756 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:19.119493    4420 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:19.119756 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:19.119533    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:27.119517074 +0000 UTC m=+16.934308343 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : configmap "kube-rbac-proxy" not found
Nov 04 03:17:19.119756 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:19.119547    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls podName:8830f068-f4c8-492e-8234-4dbf4456f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 03:17:27.119541861 +0000 UTC m=+16.934333116 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtl99" (UID: "8830f068-f4c8-492e-8234-4dbf4456f3e5") : secret "insights-runtime-extractor-tls" not found
Nov 04 03:17:19.292068 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:19.292030    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:19.360482 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:19.360440    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ckjtr" event={"ID":"36af1207-9951-4160-bb36-6dd100a05abb","Type":"ContainerStarted","Data":"9ac35b07e19974a39a26957f6243e2c29f0e20ce80d5b69b3f02bd884383eae6"}
Nov 04 03:17:19.360653 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:19.360511    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ckjtr" event={"ID":"36af1207-9951-4160-bb36-6dd100a05abb","Type":"ContainerStarted","Data":"c5b735d66dcf603f8a7d5e180eefe8f180e467feccd2bf78ace4e900ff949f94"}
Nov 04 03:17:19.360710 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:19.360689    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:19.488263 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:19.488226    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:19.690293 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:19.690246    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:19.891297 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:19.891213    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:20.094124 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:20.094089    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:20.350389 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:20.350353    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:20.499235 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:20.499198    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:20.636318 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:20.636243    4420 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-201.us-west-2.compute.internal" event="NodeReady"
Nov 04 03:17:21.294030 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:21.293998    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:21.437322 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:21.437291    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:21.584238 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:21.584161    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:21.728162 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:21.728125    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:21.891012 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:21.890928    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:22.097633 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:22.097603    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:22.494537 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:22.494496    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:23.293323 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:23.293293    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:23.492513 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:23.492484    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:23.693994 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:23.693913    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:23.890861 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:23.890807    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:24.038506 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:24.038475    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:27.169797 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:27.169759    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:27.170179 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:27.169805    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:27.170293 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:27.170271    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830f068-f4c8-492e-8234-4dbf4456f3e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:27.171951 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:27.171922    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830f068-f4c8-492e-8234-4dbf4456f3e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtl99\" (UID: \"8830f068-f4c8-492e-8234-4dbf4456f3e5\") " pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:27.262802 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:27.262771    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vtl99"
Nov 04 03:17:27.623732 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:27.623691    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vtl99"]
Nov 04 03:17:27.627574 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:27.627545    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8830f068_f4c8_492e_8234_4dbf4456f3e5.slice/crio-d3fb1d7f44b4266134753cb3ec16f539d93201651ff171baf544614b55a9e280 WatchSource:0}: Error finding container d3fb1d7f44b4266134753cb3ec16f539d93201651ff171baf544614b55a9e280: Status 404 returned error can't find the container with id d3fb1d7f44b4266134753cb3ec16f539d93201651ff171baf544614b55a9e280
Nov 04 03:17:28.390569 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:28.390530    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vtl99" event={"ID":"8830f068-f4c8-492e-8234-4dbf4456f3e5","Type":"ContainerStarted","Data":"43310504959fedf27b04c5921028915b6e468ca5306f6696df4ed88b3c50833d"}
Nov 04 03:17:28.390972 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:28.390585    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vtl99" event={"ID":"8830f068-f4c8-492e-8234-4dbf4456f3e5","Type":"ContainerStarted","Data":"d3fb1d7f44b4266134753cb3ec16f539d93201651ff171baf544614b55a9e280"}
Nov 04 03:17:29.366699 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:29.366668    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ckjtr"
Nov 04 03:17:29.395401 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:29.395369    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vtl99" event={"ID":"8830f068-f4c8-492e-8234-4dbf4456f3e5","Type":"ContainerStarted","Data":"55490b250c9a00e4b107e862a7ae315716071cb1615587bf31b8dbb0b84bee50"}
Nov 04 03:17:31.182031 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.181999    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:31.389449 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.389415    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:31.402558 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.402529    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vtl99" event={"ID":"8830f068-f4c8-492e-8234-4dbf4456f3e5","Type":"ContainerStarted","Data":"f755ed157f59a45dfc1be75fad3d441122712b8462d468f19ea9cc5f9ef52ead"}
Nov 04 03:17:31.715308 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.715227    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8645d67d9c-vvl75"]
Nov 04 03:17:31.717602 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.717581    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.803503 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.803476    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/831a88e1-95b6-48ac-a310-923e350e8894-image-registry-private-configuration\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.803503 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.803505    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/831a88e1-95b6-48ac-a310-923e350e8894-trusted-ca\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.803715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.803522    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/831a88e1-95b6-48ac-a310-923e350e8894-bound-sa-token\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.803715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.803544    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/831a88e1-95b6-48ac-a310-923e350e8894-registry-tls\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.803715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.803634    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/831a88e1-95b6-48ac-a310-923e350e8894-registry-certificates\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.803715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.803690    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/831a88e1-95b6-48ac-a310-923e350e8894-installation-pull-secrets\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.803834 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.803727    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqg7t\" (UniqueName: \"kubernetes.io/projected/831a88e1-95b6-48ac-a310-923e350e8894-kube-api-access-tqg7t\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.803834 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.803770    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/831a88e1-95b6-48ac-a310-923e350e8894-ca-trust-extracted\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.861522 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.861487    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 04 03:17:31.861522 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.861508    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 04 03:17:31.861757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.861499    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 04 03:17:31.861757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.861639    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 04 03:17:31.861757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.861710    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gl4fp\""
Nov 04 03:17:31.862521 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.862503    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8645d67d9c-vvl75"]
Nov 04 03:17:31.877003 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.876972    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:31.904948 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.904928    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/831a88e1-95b6-48ac-a310-923e350e8894-registry-tls\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.905014 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.904957    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/831a88e1-95b6-48ac-a310-923e350e8894-registry-certificates\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.905014 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.904976    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/831a88e1-95b6-48ac-a310-923e350e8894-installation-pull-secrets\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.905014 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.904995    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqg7t\" (UniqueName: \"kubernetes.io/projected/831a88e1-95b6-48ac-a310-923e350e8894-kube-api-access-tqg7t\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.905160 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.905017    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/831a88e1-95b6-48ac-a310-923e350e8894-ca-trust-extracted\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.905160 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.905045    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/831a88e1-95b6-48ac-a310-923e350e8894-image-registry-private-configuration\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.905160 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.905068    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/831a88e1-95b6-48ac-a310-923e350e8894-trusted-ca\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.905160 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.905091    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/831a88e1-95b6-48ac-a310-923e350e8894-bound-sa-token\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.905578 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.905520    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/831a88e1-95b6-48ac-a310-923e350e8894-ca-trust-extracted\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.906066 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.906043    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/831a88e1-95b6-48ac-a310-923e350e8894-registry-certificates\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.906162 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.906068    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/831a88e1-95b6-48ac-a310-923e350e8894-trusted-ca\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.907420 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.907402    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/831a88e1-95b6-48ac-a310-923e350e8894-image-registry-private-configuration\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.907603 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.907588    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/831a88e1-95b6-48ac-a310-923e350e8894-registry-tls\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.907708 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.907686    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/831a88e1-95b6-48ac-a310-923e350e8894-installation-pull-secrets\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.962452 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.962421    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqg7t\" (UniqueName: \"kubernetes.io/projected/831a88e1-95b6-48ac-a310-923e350e8894-kube-api-access-tqg7t\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:31.962892 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:31.962869    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/831a88e1-95b6-48ac-a310-923e350e8894-bound-sa-token\") pod \"image-registry-8645d67d9c-vvl75\" (UID: \"831a88e1-95b6-48ac-a310-923e350e8894\") " pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:32.026940 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:32.026907    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:32.073740 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:32.073710    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:32.370967 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:32.370876    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:32.438054 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:32.438021    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8645d67d9c-vvl75"]
Nov 04 03:17:32.441289 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:32.441266    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod831a88e1_95b6_48ac_a310_923e350e8894.slice/crio-4ba6f6ee7866ec4184a09e9cdf7667a1a269942ad3f0d8a99b3c1c9b8b2bb6bd WatchSource:0}: Error finding container 4ba6f6ee7866ec4184a09e9cdf7667a1a269942ad3f0d8a99b3c1c9b8b2bb6bd: Status 404 returned error can't find the container with id 4ba6f6ee7866ec4184a09e9cdf7667a1a269942ad3f0d8a99b3c1c9b8b2bb6bd
Nov 04 03:17:32.521305 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:32.521271    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:32.648430 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:17:32.648358    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:17:33.410342 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:33.410304    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8645d67d9c-vvl75" event={"ID":"831a88e1-95b6-48ac-a310-923e350e8894","Type":"ContainerStarted","Data":"3dbf8639bbfd4b83a229d587b10f9ba1daf8338cde965aa35854de0953ba613e"}
Nov 04 03:17:33.410731 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:33.410362    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8645d67d9c-vvl75" event={"ID":"831a88e1-95b6-48ac-a310-923e350e8894","Type":"ContainerStarted","Data":"4ba6f6ee7866ec4184a09e9cdf7667a1a269942ad3f0d8a99b3c1c9b8b2bb6bd"}
Nov 04 03:17:33.410731 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:33.410433    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:33.524732 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:33.524680    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8645d67d9c-vvl75" podStartSLOduration=2.5246669859999997 podStartE2EDuration="2.524666986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:33.524181256 +0000 UTC m=+23.338972571" watchObservedRunningTime="2026-11-04 03:17:33.524666986 +0000 UTC m=+23.339458259"
Nov 04 03:17:35.384075 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:35.384039    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:35.527222 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:35.527182    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:35.673273 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:35.673192    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:36.451184 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:36.451144    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:36.647009 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:36.646976    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:37.695429 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:37.695389    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:37.894543 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:37.894499    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:38.309092 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:38.309053    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:38.505583 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:38.505541    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:38.648681 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:38.648593    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:39.596645 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:39.596618    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vg574"
Nov 04 03:17:39.738042 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:39.738009    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:39.782809 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:39.782785    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6x62l"
Nov 04 03:17:39.888927 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:39.888830    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:40.032582 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:40.032547    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:40.291801 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:40.291770    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:40.454624 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:40.454581    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:40.601853 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:40.601766    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:40.755711 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:40.755678    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:40.908737 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:40.908659    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:41.062619 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.062562    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-gjg2b"]
Nov 04 03:17:41.066022 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.066001    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-gjg2b"
Nov 04 03:17:41.118311 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.118286    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-svns6\""
Nov 04 03:17:41.118476 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.118316    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 04 03:17:41.164372 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.164303    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ddae41c-a363-4623-ae1f-247196ee281d-tmp\") pod \"downloads-6dd789c8b7-gjg2b\" (UID: \"5ddae41c-a363-4623-ae1f-247196ee281d\") " pod="openshift-console/downloads-6dd789c8b7-gjg2b"
Nov 04 03:17:41.164372 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.164338    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkk6c\" (UniqueName: \"kubernetes.io/projected/5ddae41c-a363-4623-ae1f-247196ee281d-kube-api-access-vkk6c\") pod \"downloads-6dd789c8b7-gjg2b\" (UID: \"5ddae41c-a363-4623-ae1f-247196ee281d\") " pod="openshift-console/downloads-6dd789c8b7-gjg2b"
Nov 04 03:17:41.164950 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.164927    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 04 03:17:41.165502 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.165481    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-gjg2b"]
Nov 04 03:17:41.264659 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.264627    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ddae41c-a363-4623-ae1f-247196ee281d-tmp\") pod \"downloads-6dd789c8b7-gjg2b\" (UID: \"5ddae41c-a363-4623-ae1f-247196ee281d\") " pod="openshift-console/downloads-6dd789c8b7-gjg2b"
Nov 04 03:17:41.264659 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.264664    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkk6c\" (UniqueName: \"kubernetes.io/projected/5ddae41c-a363-4623-ae1f-247196ee281d-kube-api-access-vkk6c\") pod \"downloads-6dd789c8b7-gjg2b\" (UID: \"5ddae41c-a363-4623-ae1f-247196ee281d\") " pod="openshift-console/downloads-6dd789c8b7-gjg2b"
Nov 04 03:17:41.265065 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.265046    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ddae41c-a363-4623-ae1f-247196ee281d-tmp\") pod \"downloads-6dd789c8b7-gjg2b\" (UID: \"5ddae41c-a363-4623-ae1f-247196ee281d\") " pod="openshift-console/downloads-6dd789c8b7-gjg2b"
Nov 04 03:17:41.322128 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.322099    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkk6c\" (UniqueName: \"kubernetes.io/projected/5ddae41c-a363-4623-ae1f-247196ee281d-kube-api-access-vkk6c\") pod \"downloads-6dd789c8b7-gjg2b\" (UID: \"5ddae41c-a363-4623-ae1f-247196ee281d\") " pod="openshift-console/downloads-6dd789c8b7-gjg2b"
Nov 04 03:17:41.378817 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.378789    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-gjg2b"
Nov 04 03:17:41.489782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.489743    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:41.640784 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.640748    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:41.834165 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:41.834125    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-gjg2b"]
Nov 04 03:17:41.872899 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:41.872869    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ddae41c_a363_4623_ae1f_247196ee281d.slice/crio-a7a732ec41ca56dd3bcaa53f4315453003fd3f5f3b9b455b44af9091b47e1b69 WatchSource:0}: Error finding container a7a732ec41ca56dd3bcaa53f4315453003fd3f5f3b9b455b44af9091b47e1b69: Status 404 returned error can't find the container with id a7a732ec41ca56dd3bcaa53f4315453003fd3f5f3b9b455b44af9091b47e1b69
Nov 04 03:17:42.437421 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:42.437384    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-gjg2b" event={"ID":"5ddae41c-a363-4623-ae1f-247196ee281d","Type":"ContainerStarted","Data":"a7a732ec41ca56dd3bcaa53f4315453003fd3f5f3b9b455b44af9091b47e1b69"}
Nov 04 03:17:45.164862 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:45.164814    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:45.320147 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:45.320103    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:45.494375 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:45.494351    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5zxn5"
Nov 04 03:17:45.826586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:45.826505    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:47.167982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.167953    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:47.349145 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.349110    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:47.630917 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.630883    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:47.780120 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.780085    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:47.843290 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.843258    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"]
Nov 04 03:17:47.845716 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.845698    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:47.896629 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.896553    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 04 03:17:47.919880 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.919831    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f727f221-d129-42c3-9b3c-c44a07dff2a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-g8bvk\" (UID: \"f727f221-d129-42c3-9b3c-c44a07dff2a9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:47.920041 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.919942    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f727f221-d129-42c3-9b3c-c44a07dff2a9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-g8bvk\" (UID: \"f727f221-d129-42c3-9b3c-c44a07dff2a9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:47.920041 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.919978    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnjhc\" (UniqueName: \"kubernetes.io/projected/f727f221-d129-42c3-9b3c-c44a07dff2a9-kube-api-access-tnjhc\") pod \"openshift-state-metrics-7bd7489957-g8bvk\" (UID: \"f727f221-d129-42c3-9b3c-c44a07dff2a9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:47.920041 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.920024    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f727f221-d129-42c3-9b3c-c44a07dff2a9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-g8bvk\" (UID: \"f727f221-d129-42c3-9b3c-c44a07dff2a9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:47.942014 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.941982    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:47.944295 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.943127    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 03:17:47.944295 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.943188    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 03:17:47.944295 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.943192    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 04 03:17:47.944295 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.943207    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 03:17:47.944295 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.943269    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-427vl\""
Nov 04 03:17:47.944295 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.944225    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"]
Nov 04 03:17:47.951737 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.951716    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"]
Nov 04 03:17:47.954160 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.954142    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:47.991775 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.991737    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c25dk"]
Nov 04 03:17:47.994823 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:47.994800    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.005366 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.005345    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 04 03:17:48.005938 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.005916    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 04 03:17:48.006058 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.005976    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ck2ms\""
Nov 04 03:17:48.006372 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.006358    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 04 03:17:48.020692 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.020657    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.020819 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.020708    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.020912 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.020822    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.020912 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.020872    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f727f221-d129-42c3-9b3c-c44a07dff2a9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-g8bvk\" (UID: \"f727f221-d129-42c3-9b3c-c44a07dff2a9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:48.020912 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.020907    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnjhc\" (UniqueName: \"kubernetes.io/projected/f727f221-d129-42c3-9b3c-c44a07dff2a9-kube-api-access-tnjhc\") pod \"openshift-state-metrics-7bd7489957-g8bvk\" (UID: \"f727f221-d129-42c3-9b3c-c44a07dff2a9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:48.021045 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.020937    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.021045 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.020999    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f727f221-d129-42c3-9b3c-c44a07dff2a9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-g8bvk\" (UID: \"f727f221-d129-42c3-9b3c-c44a07dff2a9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:48.021045 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.021028    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.021149 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.021050    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f727f221-d129-42c3-9b3c-c44a07dff2a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-g8bvk\" (UID: \"f727f221-d129-42c3-9b3c-c44a07dff2a9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:48.021149 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.021068    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mpdq\" (UniqueName: \"kubernetes.io/projected/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-kube-api-access-8mpdq\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.021713 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.021688    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f727f221-d129-42c3-9b3c-c44a07dff2a9-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-g8bvk\" (UID: \"f727f221-d129-42c3-9b3c-c44a07dff2a9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:48.023898 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.023875    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f727f221-d129-42c3-9b3c-c44a07dff2a9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-g8bvk\" (UID: \"f727f221-d129-42c3-9b3c-c44a07dff2a9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:48.024025 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.024008    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f727f221-d129-42c3-9b3c-c44a07dff2a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-g8bvk\" (UID: \"f727f221-d129-42c3-9b3c-c44a07dff2a9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:48.042945 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.042918    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"]
Nov 04 03:17:48.080537 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.080511    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnjhc\" (UniqueName: \"kubernetes.io/projected/f727f221-d129-42c3-9b3c-c44a07dff2a9-kube-api-access-tnjhc\") pod \"openshift-state-metrics-7bd7489957-g8bvk\" (UID: \"f727f221-d129-42c3-9b3c-c44a07dff2a9\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:48.110364 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.110294    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 03:17:48.110534 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.110402    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ctwv8\""
Nov 04 03:17:48.110534 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.110497    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 03:17:48.110534 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.110521    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 03:17:48.121794 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.121757    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-accelerators-collector-config\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.121794 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.121798    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.122001 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.121835    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.122001 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.121905    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/91ee7662-60fe-42cf-849a-0d166cb7f5e1-root\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.122001 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.121940    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-wtmp\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.122149 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.121977    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mpdq\" (UniqueName: \"kubernetes.io/projected/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-kube-api-access-8mpdq\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.122149 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.122067    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.122149 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.122095    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-tls\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.122149 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.122110    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91ee7662-60fe-42cf-849a-0d166cb7f5e1-metrics-client-ca\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.122149 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.122128    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.122395 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.122161    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.122395 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.122176    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91ee7662-60fe-42cf-849a-0d166cb7f5e1-sys\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.122395 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.122208    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5254z\" (UniqueName: \"kubernetes.io/projected/91ee7662-60fe-42cf-849a-0d166cb7f5e1-kube-api-access-5254z\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.122395 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.122239    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.122395 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.122276    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-textfile\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.122395 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.122288    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.123360 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.123313    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.123360 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.123324    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.124755 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.124717    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.124916 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.124891    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.128075 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.128002    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:48.160868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.160788    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"
Nov 04 03:17:48.182751 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.182720    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mpdq\" (UniqueName: \"kubernetes.io/projected/5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c-kube-api-access-8mpdq\") pod \"kube-state-metrics-5dc8d7d89b-d8tbq\" (UID: \"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.223323 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.223119    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/91ee7662-60fe-42cf-849a-0d166cb7f5e1-root\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.223323 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.223162    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-wtmp\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.223323 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.223209    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-tls\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.223323 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.223233    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91ee7662-60fe-42cf-849a-0d166cb7f5e1-metrics-client-ca\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.223323 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.223238    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/91ee7662-60fe-42cf-849a-0d166cb7f5e1-root\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.223323 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.223265    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.223323 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.223309    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91ee7662-60fe-42cf-849a-0d166cb7f5e1-sys\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.223893 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.223347    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5254z\" (UniqueName: \"kubernetes.io/projected/91ee7662-60fe-42cf-849a-0d166cb7f5e1-kube-api-access-5254z\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.223893 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.223378    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-wtmp\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.223893 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.223386    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-textfile\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.223893 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.223453    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-accelerators-collector-config\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.224108 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.224067    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91ee7662-60fe-42cf-849a-0d166cb7f5e1-sys\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.226769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.226740    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.227140 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.227111    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-textfile\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.227602 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.227578    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-accelerators-collector-config\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.228040 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.228020    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91ee7662-60fe-42cf-849a-0d166cb7f5e1-metrics-client-ca\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.228707 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.228676    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/91ee7662-60fe-42cf-849a-0d166cb7f5e1-node-exporter-tls\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.268077 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.268046    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"
Nov 04 03:17:48.282043 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.282014    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5254z\" (UniqueName: \"kubernetes.io/projected/91ee7662-60fe-42cf-849a-0d166cb7f5e1-kube-api-access-5254z\") pod \"node-exporter-c25dk\" (UID: \"91ee7662-60fe-42cf-849a-0d166cb7f5e1\") " pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.301031 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.301005    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:48.310583 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.310554    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c25dk"
Nov 04 03:17:48.454382 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.454307    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:48.458016 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.457984    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c25dk" event={"ID":"91ee7662-60fe-42cf-849a-0d166cb7f5e1","Type":"ContainerStarted","Data":"51ace273e8acf001d4b5c0655385a68cc407d5974b8efa32492935166952e4e2"}
Nov 04 03:17:48.586319 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.586280    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk"]
Nov 04 03:17:48.590761 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:48.590728    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf727f221_d129_42c3_9b3c_c44a07dff2a9.slice/crio-b147eae2fa5555bbb79780695c0405156553a375ff5de428bed273ba9e147d75 WatchSource:0}: Error finding container b147eae2fa5555bbb79780695c0405156553a375ff5de428bed273ba9e147d75: Status 404 returned error can't find the container with id b147eae2fa5555bbb79780695c0405156553a375ff5de428bed273ba9e147d75
Nov 04 03:17:48.637725 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.637699    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq"]
Nov 04 03:17:48.641158 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:48.641125    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd4d229_f69b_4f92_9f92_5f6fe1e3b42c.slice/crio-c2aee94f580114c2a4a79c25ad2a8b2c6909cc3fab442211123cc97f264f70b8 WatchSource:0}: Error finding container c2aee94f580114c2a4a79c25ad2a8b2c6909cc3fab442211123cc97f264f70b8: Status 404 returned error can't find the container with id c2aee94f580114c2a4a79c25ad2a8b2c6909cc3fab442211123cc97f264f70b8
Nov 04 03:17:48.767458 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:48.767418    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:49.270181 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:49.270150    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:49.462995 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:49.462954    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq" event={"ID":"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c","Type":"ContainerStarted","Data":"c2aee94f580114c2a4a79c25ad2a8b2c6909cc3fab442211123cc97f264f70b8"}
Nov 04 03:17:49.465001 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:49.464974    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk" event={"ID":"f727f221-d129-42c3-9b3c-c44a07dff2a9","Type":"ContainerStarted","Data":"cbf2f6cfa549b7b3b00ce09bac1bd6cdbc382cec9e5bec9e1901fe87a6d9e72c"}
Nov 04 03:17:49.465131 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:49.465032    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk" event={"ID":"f727f221-d129-42c3-9b3c-c44a07dff2a9","Type":"ContainerStarted","Data":"0088f7c451e63156560b86f038151e6d78dad44feb3f0bc324b0f2c1309309b0"}
Nov 04 03:17:49.465131 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:49.465054    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk" event={"ID":"f727f221-d129-42c3-9b3c-c44a07dff2a9","Type":"ContainerStarted","Data":"b147eae2fa5555bbb79780695c0405156553a375ff5de428bed273ba9e147d75"}
Nov 04 03:17:49.498235 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:49.498204    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:50.170305 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:50.170239    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:51.483295 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:51.483195    4420 generic.go:356] "Generic (PLEG): container finished" podID="91ee7662-60fe-42cf-849a-0d166cb7f5e1" containerID="df8dd9f90a68e116cc3ff379e8bd0cd3035ac74adc5983f0ec639638115a372b" exitCode=0
Nov 04 03:17:51.484186 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:51.484150    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c25dk" event={"ID":"91ee7662-60fe-42cf-849a-0d166cb7f5e1","Type":"ContainerDied","Data":"df8dd9f90a68e116cc3ff379e8bd0cd3035ac74adc5983f0ec639638115a372b"}
Nov 04 03:17:51.489508 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:51.489470    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq" event={"ID":"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c","Type":"ContainerStarted","Data":"4bb7855add6032b3573c9701fad767cc58637c63fd8bcbea979782fae2542bb5"}
Nov 04 03:17:51.489625 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:51.489541    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq" event={"ID":"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c","Type":"ContainerStarted","Data":"d555378ebfbc3e62f77f8fb2b820e1389645e3c84a7ffbbf316581628bc39127"}
Nov 04 03:17:51.489625 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:51.489577    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq" event={"ID":"5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c","Type":"ContainerStarted","Data":"a13bfc41a1cb6e415c60dfcb007f5e4f520ae36b6cc3d96d41a94868cd4a2554"}
Nov 04 03:17:51.492796 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:51.492772    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk" event={"ID":"f727f221-d129-42c3-9b3c-c44a07dff2a9","Type":"ContainerStarted","Data":"f910e7763d7df93870a0f8be06df1bc7414a8db9ca3492449245484ba627eb46"}
Nov 04 03:17:51.756199 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:51.756143    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-g8bvk" podStartSLOduration=2.7948205980000003 podStartE2EDuration="4.756128959s" totalImagesPullingTime="1.961308361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:51.755798558 +0000 UTC m=+41.570589836" watchObservedRunningTime="2026-11-04 03:17:51.756128959 +0000 UTC m=+41.570920233"
Nov 04 03:17:51.870031 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:51.869978    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-d8tbq" podStartSLOduration=2.7199150789999997 podStartE2EDuration="4.869960653s" totalImagesPullingTime="2.150045574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:51.869161381 +0000 UTC m=+41.683952659" watchObservedRunningTime="2026-11-04 03:17:51.869960653 +0000 UTC m=+41.684751932"
Nov 04 03:17:52.070278 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.070181    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:52.225523 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.225487    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:52.468959 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.468890    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:52.499428 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.499391    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c25dk" event={"ID":"91ee7662-60fe-42cf-849a-0d166cb7f5e1","Type":"ContainerStarted","Data":"3903dc107c20e4887114e8cca3c867946e6c69914531da078036dbcb24d9e530"}
Nov 04 03:17:52.499876 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.499453    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c25dk" event={"ID":"91ee7662-60fe-42cf-849a-0d166cb7f5e1","Type":"ContainerStarted","Data":"dec7bacc2e451bc36d6220da82e9204cfa96fee98923b8787711cd059b1bcd81"}
Nov 04 03:17:52.568457 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.568421    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"]
Nov 04 03:17:52.571894 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.571870    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"
Nov 04 03:17:52.629527 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.629455    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c25dk" podStartSLOduration=3.3969352 podStartE2EDuration="5.629438489s" totalImagesPullingTime="2.147326188s" totalInitContainerRuntime="85.177101ms" isStatefulPod=true podCreationTimestamp="2026-11-04 03:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:52.620556158 +0000 UTC m=+42.435347436" watchObservedRunningTime="2026-11-04 03:17:52.629438489 +0000 UTC m=+42.444229767"
Nov 04 03:17:52.630168 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.630146    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"]
Nov 04 03:17:52.634664 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.634575    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:52.662383 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.662353    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hb78\" (UniqueName: \"kubernetes.io/projected/082d366a-f1ca-4eba-a55d-d1b683942520-kube-api-access-7hb78\") pod \"monitoring-plugin-77f4f76594-mrzfr\" (UID: \"082d366a-f1ca-4eba-a55d-d1b683942520\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"
Nov 04 03:17:52.662536 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.662417    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/082d366a-f1ca-4eba-a55d-d1b683942520-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-mrzfr\" (UID: \"082d366a-f1ca-4eba-a55d-d1b683942520\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"
Nov 04 03:17:52.665303 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.665280    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vplqx\""
Nov 04 03:17:52.665303 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.665278    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 04 03:17:52.763718 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.763685    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/082d366a-f1ca-4eba-a55d-d1b683942520-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-mrzfr\" (UID: \"082d366a-f1ca-4eba-a55d-d1b683942520\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"
Nov 04 03:17:52.763919 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.763768    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hb78\" (UniqueName: \"kubernetes.io/projected/082d366a-f1ca-4eba-a55d-d1b683942520-kube-api-access-7hb78\") pod \"monitoring-plugin-77f4f76594-mrzfr\" (UID: \"082d366a-f1ca-4eba-a55d-d1b683942520\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"
Nov 04 03:17:52.766595 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.766569    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/082d366a-f1ca-4eba-a55d-d1b683942520-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-mrzfr\" (UID: \"082d366a-f1ca-4eba-a55d-d1b683942520\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"
Nov 04 03:17:52.836863 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.836756    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:52.837516 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.837492    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hb78\" (UniqueName: \"kubernetes.io/projected/082d366a-f1ca-4eba-a55d-d1b683942520-kube-api-access-7hb78\") pod \"monitoring-plugin-77f4f76594-mrzfr\" (UID: \"082d366a-f1ca-4eba-a55d-d1b683942520\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"
Nov 04 03:17:52.885442 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.885412    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"
Nov 04 03:17:52.988966 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:52.988929    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:53.254465 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:53.254391    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"]
Nov 04 03:17:53.258724 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:53.258698    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod082d366a_f1ca_4eba_a55d_d1b683942520.slice/crio-8f6d7f6057e0576c75c7da3e4cab289dfbd157e94d6c865ded53280826e6b103 WatchSource:0}: Error finding container 8f6d7f6057e0576c75c7da3e4cab289dfbd157e94d6c865ded53280826e6b103: Status 404 returned error can't find the container with id 8f6d7f6057e0576c75c7da3e4cab289dfbd157e94d6c865ded53280826e6b103
Nov 04 03:17:53.504732 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:53.504645    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr" event={"ID":"082d366a-f1ca-4eba-a55d-d1b683942520","Type":"ContainerStarted","Data":"8f6d7f6057e0576c75c7da3e4cab289dfbd157e94d6c865ded53280826e6b103"}
Nov 04 03:17:54.418591 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:54.418555    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8645d67d9c-vvl75"
Nov 04 03:17:54.748276 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:54.748228    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:54.901541 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:54.901504    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:55.051573 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:55.051492    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:55.196049 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:55.196009    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:56.744765 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:56.744726    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:56.889330 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:56.889300    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:57.099785 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.099689    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:57.175500 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.175466    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 03:17:57.179212 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.179189    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.275950 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.275824    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 04 03:17:57.275950 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.275898    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 04 03:17:57.275950 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.275915    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 04 03:17:57.275950 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.275922    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 04 03:17:57.275950 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.275923    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 04 03:17:57.275950 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.275920    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 04 03:17:57.275950 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.275958    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 04 03:17:57.276363 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.276104    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-15iq5iq7eocur\""
Nov 04 03:17:57.276363 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.276161    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 04 03:17:57.276363 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.276272    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kr76c\""
Nov 04 03:17:57.304104 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304075    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304268 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304112    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304268 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304144    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304268 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304205    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-web-config\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304429 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304288    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-config-out\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304429 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304326    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304429 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304352    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304429 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304371    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd4p4\" (UniqueName: \"kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-kube-api-access-hd4p4\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304429 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304389    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304429 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304405    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304429 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304427    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304451    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304486    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304504    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304523    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-config\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304545    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304572    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304593    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304623    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.304758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.304646    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.327364 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.325009    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 04 03:17:57.327364 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.325904    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 03:17:57.329973 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.329947    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 04 03:17:57.330079 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.329987    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 04 03:17:57.330079 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.330021    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 04 03:17:57.330195 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.330088    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 03:17:57.330195 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.330145    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 04 03:17:57.370166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.370100    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 04 03:17:57.405970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.405938    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406120 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.405985    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406120 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406017    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406120 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406044    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406120 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406073    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406120 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406100    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406354 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406230    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-config\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406354 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406261    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406354 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406289    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406354 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406314    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406354 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406348    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406592 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406382    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406592 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406428    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406592 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406463    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406592 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406492    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406592 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406508    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-web-config\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406592 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406535    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-config-out\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406592 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406574    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.406592 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406592    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.407026 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406608    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd4p4\" (UniqueName: \"kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-kube-api-access-hd4p4\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.407026 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.406781    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.407131 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.407085    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.407659 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.407288    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.412600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.408158    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.412600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.408329    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.412600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.408717    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.412600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.410020    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.412600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.410075    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.412600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.410437    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-config\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.412600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.410494    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.412600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.411331    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.412600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.411854    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.412600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.412226    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.412600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.412292    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-web-config\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.412600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.412564    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.413324 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.413302    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-config-out\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.413442 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.413426    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.413510 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.413473    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.414667 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.414649    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.464757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.464725    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd4p4\" (UniqueName: \"kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-kube-api-access-hd4p4\") pod \"prometheus-k8s-0\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.505477 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.505441    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:17:57.522820 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.522776    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr" event={"ID":"082d366a-f1ca-4eba-a55d-d1b683942520","Type":"ContainerStarted","Data":"be9fd6bc1ad84121fa16a889912473b121a99f6c83c6cebcc95bd86c35473bda"}
Nov 04 03:17:57.523078 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.523019    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"
Nov 04 03:17:57.528744 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.528692    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr"
Nov 04 03:17:57.632567 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.632467    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mrzfr" podStartSLOduration=2.136181139 podStartE2EDuration="5.632452147s" totalImagesPullingTime="3.496271008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:57.63230508 +0000 UTC m=+47.447096356" watchObservedRunningTime="2026-11-04 03:17:57.632452147 +0000 UTC m=+47.447243425"
Nov 04 03:17:57.770598 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.770548    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:17:57.943284 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:57.943207    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 03:17:57.950939 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:17:57.950905    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod667f8d76_bee3_4b64_afb7_59c789573285.slice/crio-8974dad8af0497705714d6eb5d665626c79ee600b3b225fe1389f7643e5a70b1 WatchSource:0}: Error finding container 8974dad8af0497705714d6eb5d665626c79ee600b3b225fe1389f7643e5a70b1: Status 404 returned error can't find the container with id 8974dad8af0497705714d6eb5d665626c79ee600b3b225fe1389f7643e5a70b1
Nov 04 03:17:58.527808 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:58.527776    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerStarted","Data":"8974dad8af0497705714d6eb5d665626c79ee600b3b225fe1389f7643e5a70b1"}
Nov 04 03:17:59.535335 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:59.535288    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-gjg2b" event={"ID":"5ddae41c-a363-4623-ae1f-247196ee281d","Type":"ContainerStarted","Data":"d50fe8b4979ff8538233f0a443bdca902a3b658f002a753423b521a397a4d989"}
Nov 04 03:17:59.535833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:59.535622    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-gjg2b"
Nov 04 03:17:59.552486 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:59.552457    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-gjg2b"
Nov 04 03:17:59.646202 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:59.646147    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-gjg2b" podStartSLOduration=1.842798658 podStartE2EDuration="18.646130762s" totalImagesPullingTime="16.803332104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:17:59.645538701 +0000 UTC m=+49.460330002" watchObservedRunningTime="2026-11-04 03:17:59.646130762 +0000 UTC m=+49.460922039"
Nov 04 03:17:59.931253 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:17:59.931212    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:00.107365 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:00.107279    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:00.252544 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:00.252504    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:00.541243 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:00.541203    4420 generic.go:356] "Generic (PLEG): container finished" podID="667f8d76-bee3-4b64-afb7-59c789573285" containerID="57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc" exitCode=0
Nov 04 03:18:00.541714 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:00.541311    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerDied","Data":"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"}
Nov 04 03:18:01.184622 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:01.184582    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:01.330649 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:01.330612    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:02.656980 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:18:02.656920    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:18:05.565397 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:05.565357    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerStarted","Data":"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"}
Nov 04 03:18:05.565971 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:05.565484    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerStarted","Data":"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"}
Nov 04 03:18:09.584460 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:09.584421    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerStarted","Data":"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"}
Nov 04 03:18:09.584889 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:09.584502    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerStarted","Data":"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"}
Nov 04 03:18:09.584889 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:09.584546    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerStarted","Data":"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"}
Nov 04 03:18:09.584889 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:09.584584    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerStarted","Data":"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"}
Nov 04 03:18:09.848775 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:09.848679    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.360480357 podStartE2EDuration="12.848665982s" totalImagesPullingTime="9.467404697s" totalInitContainerRuntime="20.780928ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:17:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:18:09.801457911 +0000 UTC m=+59.616249187" watchObservedRunningTime="2026-11-04 03:18:09.848665982 +0000 UTC m=+59.663457258"
Nov 04 03:18:10.078661 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:10.078625    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:10.227979 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:10.227904    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:10.375565 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:10.375520    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:11.178707 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:11.178660    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:11.337088 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:11.337057    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:12.506026 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:12.505991    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:18:16.179195 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:16.179156    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:16.374860 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:16.374819    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:31.190272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:31.190230    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:31.389534 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:31.389497    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:32.649185 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:18:32.649148    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:18:41.263547 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:41.263508    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:42.123765 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:42.123723    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:18:46.948357 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:46.948313    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:18:46.967102 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:46.967076    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:18:47.521205 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:18:47.521178    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:00.292222 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:00.292182    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:01.805143 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:01.805111    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:02.058710 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:02.058611    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:02.647227 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:19:02.647189    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:19:02.849880 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:02.849828    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:09.840366 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:09.840336    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 03:19:09.840769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:09.840611    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="prometheus" containerID="cri-o://4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725" gracePeriod=600
Nov 04 03:19:09.840769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:09.840631    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="kube-rbac-proxy" containerID="cri-o://e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d" gracePeriod=600
Nov 04 03:19:09.840769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:09.840663    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="kube-rbac-proxy-thanos" containerID="cri-o://95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27" gracePeriod=600
Nov 04 03:19:09.840769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:09.840680    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="thanos-sidecar" containerID="cri-o://451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce" gracePeriod=600
Nov 04 03:19:09.840769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:09.840691    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="config-reloader" containerID="cri-o://187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396" gracePeriod=600
Nov 04 03:19:09.840769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:09.840735    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="kube-rbac-proxy-web" containerID="cri-o://b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d" gracePeriod=600
Nov 04 03:19:10.079678 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.079656    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:10.174141 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174067    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.174141 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174118    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-metrics-client-ca\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.174141 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174139    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.174392 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174173    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-db\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.174392 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174195    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-kube-rbac-proxy\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.174392 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174229    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-thanos-prometheus-http-client-file\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.174392 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174255    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-kubelet-serving-ca-bundle\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.174392 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174284    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-config-out\" (UniqueName: \"kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-config-out\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.174654 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174523    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-metrics-client-ca" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:10.174654 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174596    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-grpc-tls\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.174771 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174734    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-kube-api-access-hd4p4\" (UniqueName: \"kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-kube-api-access-hd4p4\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.174825 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174785    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-web-config\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-web-config\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.174825 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174815    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-metrics-client-certs\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.174971 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174819    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-kubelet-serving-ca-bundle" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:10.174971 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.174875    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-trusted-ca-bundle\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.175483 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.175222    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-trusted-ca-bundle" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:10.175607 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.175555    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-db" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:19:10.175681 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.175623    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-2\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.175681 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.175663    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-config\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-config\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.176012 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.175703    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-tls-assets\" (UniqueName: \"kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-tls-assets\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.176012 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.175741    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-1\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.176012 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.175779    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-tls\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.176012 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.175900    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-serving-certs-ca-bundle\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.176012 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.175931    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-0\") pod \"667f8d76-bee3-4b64-afb7-59c789573285\" (UID: \"667f8d76-bee3-4b64-afb7-59c789573285\") "
Nov 04 03:19:10.176012 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.175971    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-2" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:10.176343 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.176168    4420 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-db\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.176343 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.176187    4420 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.176343 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.176203    4420 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.176343 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.176218    4420 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.176343 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.176233    4420 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-metrics-client-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.176343 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.176252    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-1" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:10.177434 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.177311    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-kube-api-access-hd4p4" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "kube-api-access-hd4p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:19:10.177537 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.177437    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-kube-rbac-proxy-web" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:10.177537 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.177480    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-config-out" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:19:10.177772 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.177736    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-serving-certs-ca-bundle" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:10.177945 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.177813    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-0" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:19:10.178024 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.177984    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-thanos-prometheus-http-client-file" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:10.178197 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.178159    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-thanos-sidecar-tls" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:10.178392 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.178360    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-metrics-client-certs" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:10.178487 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.178390    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-kube-rbac-proxy" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:10.179535 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.179514    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-config" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:10.179657 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.179638    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-grpc-tls" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:10.179709 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.179654    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-tls" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:10.179709 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.179677    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-tls-assets" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:19:10.188478 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.188451    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-web-config" pod "667f8d76-bee3-4b64-afb7-59c789573285" (UID: "667f8d76-bee3-4b64-afb7-59c789573285"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:19:10.277076 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277050    4420 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277076 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277074    4420 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/667f8d76-bee3-4b64-afb7-59c789573285-config-out\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277076 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277083    4420 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-grpc-tls\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277092    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd4p4\" (UniqueName: \"kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-kube-api-access-hd4p4\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277101    4420 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-web-config\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277110    4420 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-metrics-client-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277118    4420 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-config\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277126    4420 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/667f8d76-bee3-4b64-afb7-59c789573285-tls-assets\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277134    4420 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277143    4420 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277151    4420 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277160    4420 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/667f8d76-bee3-4b64-afb7-59c789573285-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277168    4420 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277177    4420 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.277272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.277187    4420 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/667f8d76-bee3-4b64-afb7-59c789573285-secret-kube-rbac-proxy\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:19:10.791335 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791301    4420 generic.go:356] "Generic (PLEG): container finished" podID="667f8d76-bee3-4b64-afb7-59c789573285" containerID="95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27" exitCode=0
Nov 04 03:19:10.791335 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791328    4420 generic.go:356] "Generic (PLEG): container finished" podID="667f8d76-bee3-4b64-afb7-59c789573285" containerID="e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d" exitCode=0
Nov 04 03:19:10.791335 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791337    4420 generic.go:356] "Generic (PLEG): container finished" podID="667f8d76-bee3-4b64-afb7-59c789573285" containerID="b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d" exitCode=0
Nov 04 03:19:10.791335 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791343    4420 generic.go:356] "Generic (PLEG): container finished" podID="667f8d76-bee3-4b64-afb7-59c789573285" containerID="451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce" exitCode=0
Nov 04 03:19:10.791590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791352    4420 generic.go:356] "Generic (PLEG): container finished" podID="667f8d76-bee3-4b64-afb7-59c789573285" containerID="187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396" exitCode=0
Nov 04 03:19:10.791590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791358    4420 generic.go:356] "Generic (PLEG): container finished" podID="667f8d76-bee3-4b64-afb7-59c789573285" containerID="4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725" exitCode=0
Nov 04 03:19:10.791590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791390    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerDied","Data":"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"}
Nov 04 03:19:10.791590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791457    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerDied","Data":"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"}
Nov 04 03:19:10.791590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791401    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:10.791590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791486    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerDied","Data":"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"}
Nov 04 03:19:10.791590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791512    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerDied","Data":"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"}
Nov 04 03:19:10.791590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791535    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerDied","Data":"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"}
Nov 04 03:19:10.791590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791556    4420 scope.go:122] "RemoveContainer" containerID="95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"
Nov 04 03:19:10.791590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791560    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerDied","Data":"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"}
Nov 04 03:19:10.791590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.791583    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"667f8d76-bee3-4b64-afb7-59c789573285","Type":"ContainerDied","Data":"8974dad8af0497705714d6eb5d665626c79ee600b3b225fe1389f7643e5a70b1"}
Nov 04 03:19:10.798925 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.798908    4420 scope.go:122] "RemoveContainer" containerID="e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"
Nov 04 03:19:10.805263 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.805247    4420 scope.go:122] "RemoveContainer" containerID="b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"
Nov 04 03:19:10.811455 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.811439    4420 scope.go:122] "RemoveContainer" containerID="451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"
Nov 04 03:19:10.817405 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.817390    4420 scope.go:122] "RemoveContainer" containerID="187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"
Nov 04 03:19:10.823347 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.823332    4420 scope.go:122] "RemoveContainer" containerID="4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"
Nov 04 03:19:10.829659 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.829643    4420 scope.go:122] "RemoveContainer" containerID="57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"
Nov 04 03:19:10.835554 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.835539    4420 scope.go:122] "RemoveContainer" containerID="95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"
Nov 04 03:19:10.835801 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:19:10.835781    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": container with ID starting with 95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27 not found: ID does not exist" containerID="95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"
Nov 04 03:19:10.835864 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.835809    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"} err="failed to get container status \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": rpc error: code = NotFound desc = could not find container \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": container with ID starting with 95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27 not found: ID does not exist"
Nov 04 03:19:10.835864 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.835851    4420 scope.go:122] "RemoveContainer" containerID="e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"
Nov 04 03:19:10.836122 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:19:10.836104    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": container with ID starting with e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d not found: ID does not exist" containerID="e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"
Nov 04 03:19:10.836170 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.836127    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"} err="failed to get container status \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": rpc error: code = NotFound desc = could not find container \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": container with ID starting with e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d not found: ID does not exist"
Nov 04 03:19:10.836170 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.836142    4420 scope.go:122] "RemoveContainer" containerID="b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"
Nov 04 03:19:10.836364 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:19:10.836345    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": container with ID starting with b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d not found: ID does not exist" containerID="b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"
Nov 04 03:19:10.836403 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.836368    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"} err="failed to get container status \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": rpc error: code = NotFound desc = could not find container \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": container with ID starting with b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d not found: ID does not exist"
Nov 04 03:19:10.836403 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.836379    4420 scope.go:122] "RemoveContainer" containerID="451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"
Nov 04 03:19:10.836603 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:19:10.836587    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": container with ID starting with 451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce not found: ID does not exist" containerID="451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"
Nov 04 03:19:10.836669 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.836611    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"} err="failed to get container status \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": rpc error: code = NotFound desc = could not find container \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": container with ID starting with 451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce not found: ID does not exist"
Nov 04 03:19:10.836669 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.836630    4420 scope.go:122] "RemoveContainer" containerID="187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"
Nov 04 03:19:10.836878 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:19:10.836861    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": container with ID starting with 187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396 not found: ID does not exist" containerID="187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"
Nov 04 03:19:10.836926 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.836885    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"} err="failed to get container status \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": rpc error: code = NotFound desc = could not find container \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": container with ID starting with 187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396 not found: ID does not exist"
Nov 04 03:19:10.836926 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.836897    4420 scope.go:122] "RemoveContainer" containerID="4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"
Nov 04 03:19:10.837116 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:19:10.837100    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": container with ID starting with 4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725 not found: ID does not exist" containerID="4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"
Nov 04 03:19:10.837158 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.837121    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"} err="failed to get container status \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": rpc error: code = NotFound desc = could not find container \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": container with ID starting with 4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725 not found: ID does not exist"
Nov 04 03:19:10.837158 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.837134    4420 scope.go:122] "RemoveContainer" containerID="57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"
Nov 04 03:19:10.837341 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:19:10.837327    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": container with ID starting with 57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc not found: ID does not exist" containerID="57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"
Nov 04 03:19:10.837380 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.837344    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"} err="failed to get container status \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": rpc error: code = NotFound desc = could not find container \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": container with ID starting with 57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc not found: ID does not exist"
Nov 04 03:19:10.837380 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.837356    4420 scope.go:122] "RemoveContainer" containerID="95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"
Nov 04 03:19:10.837556 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.837538    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"} err="failed to get container status \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": rpc error: code = NotFound desc = could not find container \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": container with ID starting with 95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27 not found: ID does not exist"
Nov 04 03:19:10.837624 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.837557    4420 scope.go:122] "RemoveContainer" containerID="e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"
Nov 04 03:19:10.837764 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.837748    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"} err="failed to get container status \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": rpc error: code = NotFound desc = could not find container \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": container with ID starting with e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d not found: ID does not exist"
Nov 04 03:19:10.837807 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.837763    4420 scope.go:122] "RemoveContainer" containerID="b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"
Nov 04 03:19:10.838219 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.838201    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"} err="failed to get container status \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": rpc error: code = NotFound desc = could not find container \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": container with ID starting with b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d not found: ID does not exist"
Nov 04 03:19:10.838293 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.838220    4420 scope.go:122] "RemoveContainer" containerID="451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"
Nov 04 03:19:10.838435 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.838419    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"} err="failed to get container status \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": rpc error: code = NotFound desc = could not find container \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": container with ID starting with 451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce not found: ID does not exist"
Nov 04 03:19:10.838479 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.838435    4420 scope.go:122] "RemoveContainer" containerID="187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"
Nov 04 03:19:10.838623 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.838606    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"} err="failed to get container status \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": rpc error: code = NotFound desc = could not find container \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": container with ID starting with 187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396 not found: ID does not exist"
Nov 04 03:19:10.838688 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.838623    4420 scope.go:122] "RemoveContainer" containerID="4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"
Nov 04 03:19:10.838815 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.838799    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"} err="failed to get container status \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": rpc error: code = NotFound desc = could not find container \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": container with ID starting with 4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725 not found: ID does not exist"
Nov 04 03:19:10.838872 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.838815    4420 scope.go:122] "RemoveContainer" containerID="57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"
Nov 04 03:19:10.839048 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.839030    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"} err="failed to get container status \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": rpc error: code = NotFound desc = could not find container \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": container with ID starting with 57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc not found: ID does not exist"
Nov 04 03:19:10.839117 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.839049    4420 scope.go:122] "RemoveContainer" containerID="95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"
Nov 04 03:19:10.839274 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.839256    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"} err="failed to get container status \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": rpc error: code = NotFound desc = could not find container \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": container with ID starting with 95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27 not found: ID does not exist"
Nov 04 03:19:10.839322 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.839273    4420 scope.go:122] "RemoveContainer" containerID="e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"
Nov 04 03:19:10.839473 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.839455    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"} err="failed to get container status \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": rpc error: code = NotFound desc = could not find container \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": container with ID starting with e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d not found: ID does not exist"
Nov 04 03:19:10.839535 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.839474    4420 scope.go:122] "RemoveContainer" containerID="b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"
Nov 04 03:19:10.839664 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.839648    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"} err="failed to get container status \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": rpc error: code = NotFound desc = could not find container \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": container with ID starting with b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d not found: ID does not exist"
Nov 04 03:19:10.839728 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.839664    4420 scope.go:122] "RemoveContainer" containerID="451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"
Nov 04 03:19:10.839907 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.839889    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"} err="failed to get container status \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": rpc error: code = NotFound desc = could not find container \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": container with ID starting with 451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce not found: ID does not exist"
Nov 04 03:19:10.839959 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.839909    4420 scope.go:122] "RemoveContainer" containerID="187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"
Nov 04 03:19:10.840127 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.840108    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"} err="failed to get container status \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": rpc error: code = NotFound desc = could not find container \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": container with ID starting with 187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396 not found: ID does not exist"
Nov 04 03:19:10.840170 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.840128    4420 scope.go:122] "RemoveContainer" containerID="4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"
Nov 04 03:19:10.840362 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.840344    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"} err="failed to get container status \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": rpc error: code = NotFound desc = could not find container \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": container with ID starting with 4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725 not found: ID does not exist"
Nov 04 03:19:10.840428 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.840363    4420 scope.go:122] "RemoveContainer" containerID="57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"
Nov 04 03:19:10.840676 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.840543    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"} err="failed to get container status \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": rpc error: code = NotFound desc = could not find container \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": container with ID starting with 57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc not found: ID does not exist"
Nov 04 03:19:10.840676 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.840558    4420 scope.go:122] "RemoveContainer" containerID="95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"
Nov 04 03:19:10.840764 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.840749    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"} err="failed to get container status \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": rpc error: code = NotFound desc = could not find container \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": container with ID starting with 95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27 not found: ID does not exist"
Nov 04 03:19:10.840801 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.840764    4420 scope.go:122] "RemoveContainer" containerID="e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"
Nov 04 03:19:10.840977 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.840958    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"} err="failed to get container status \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": rpc error: code = NotFound desc = could not find container \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": container with ID starting with e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d not found: ID does not exist"
Nov 04 03:19:10.841037 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.840979    4420 scope.go:122] "RemoveContainer" containerID="b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"
Nov 04 03:19:10.841168 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.841152    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"} err="failed to get container status \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": rpc error: code = NotFound desc = could not find container \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": container with ID starting with b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d not found: ID does not exist"
Nov 04 03:19:10.841215 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.841168    4420 scope.go:122] "RemoveContainer" containerID="451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"
Nov 04 03:19:10.841364 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.841346    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"} err="failed to get container status \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": rpc error: code = NotFound desc = could not find container \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": container with ID starting with 451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce not found: ID does not exist"
Nov 04 03:19:10.841429 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.841365    4420 scope.go:122] "RemoveContainer" containerID="187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"
Nov 04 03:19:10.841573 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.841557    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"} err="failed to get container status \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": rpc error: code = NotFound desc = could not find container \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": container with ID starting with 187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396 not found: ID does not exist"
Nov 04 03:19:10.841624 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.841573    4420 scope.go:122] "RemoveContainer" containerID="4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"
Nov 04 03:19:10.841756 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.841737    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"} err="failed to get container status \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": rpc error: code = NotFound desc = could not find container \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": container with ID starting with 4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725 not found: ID does not exist"
Nov 04 03:19:10.841797 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.841756    4420 scope.go:122] "RemoveContainer" containerID="57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"
Nov 04 03:19:10.841987 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.841967    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"} err="failed to get container status \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": rpc error: code = NotFound desc = could not find container \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": container with ID starting with 57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc not found: ID does not exist"
Nov 04 03:19:10.842042 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.841988    4420 scope.go:122] "RemoveContainer" containerID="95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"
Nov 04 03:19:10.842214 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.842195    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"} err="failed to get container status \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": rpc error: code = NotFound desc = could not find container \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": container with ID starting with 95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27 not found: ID does not exist"
Nov 04 03:19:10.842288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.842215    4420 scope.go:122] "RemoveContainer" containerID="e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"
Nov 04 03:19:10.842456 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.842427    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"} err="failed to get container status \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": rpc error: code = NotFound desc = could not find container \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": container with ID starting with e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d not found: ID does not exist"
Nov 04 03:19:10.842456 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.842452    4420 scope.go:122] "RemoveContainer" containerID="b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"
Nov 04 03:19:10.842697 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.842681    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"} err="failed to get container status \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": rpc error: code = NotFound desc = could not find container \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": container with ID starting with b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d not found: ID does not exist"
Nov 04 03:19:10.842772 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.842700    4420 scope.go:122] "RemoveContainer" containerID="451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"
Nov 04 03:19:10.843045 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.843021    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"} err="failed to get container status \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": rpc error: code = NotFound desc = could not find container \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": container with ID starting with 451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce not found: ID does not exist"
Nov 04 03:19:10.843128 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.843047    4420 scope.go:122] "RemoveContainer" containerID="187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"
Nov 04 03:19:10.843298 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.843274    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"} err="failed to get container status \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": rpc error: code = NotFound desc = could not find container \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": container with ID starting with 187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396 not found: ID does not exist"
Nov 04 03:19:10.843298 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.843297    4420 scope.go:122] "RemoveContainer" containerID="4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"
Nov 04 03:19:10.844815 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.844795    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"} err="failed to get container status \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": rpc error: code = NotFound desc = could not find container \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": container with ID starting with 4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725 not found: ID does not exist"
Nov 04 03:19:10.844952 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.844816    4420 scope.go:122] "RemoveContainer" containerID="57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"
Nov 04 03:19:10.845089 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.845071    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"} err="failed to get container status \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": rpc error: code = NotFound desc = could not find container \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": container with ID starting with 57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc not found: ID does not exist"
Nov 04 03:19:10.845129 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.845090    4420 scope.go:122] "RemoveContainer" containerID="95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"
Nov 04 03:19:10.845309 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.845291    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27"} err="failed to get container status \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": rpc error: code = NotFound desc = could not find container \"95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27\": container with ID starting with 95b73738d0fa567ce7e8fa2b6bc4b96d8454d4371b0542567cf2675eb6571e27 not found: ID does not exist"
Nov 04 03:19:10.845345 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.845310    4420 scope.go:122] "RemoveContainer" containerID="e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"
Nov 04 03:19:10.845542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.845527    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d"} err="failed to get container status \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": rpc error: code = NotFound desc = could not find container \"e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d\": container with ID starting with e32117e8d03b5085107aac641eef33f9ac6796360d27f04fe530707222a6a68d not found: ID does not exist"
Nov 04 03:19:10.845575 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.845543    4420 scope.go:122] "RemoveContainer" containerID="b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"
Nov 04 03:19:10.845750 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.845735    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d"} err="failed to get container status \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": rpc error: code = NotFound desc = could not find container \"b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d\": container with ID starting with b07f55f482528f62ea8a09a8fb27d9b24b4a655df7e40e9359c43bd6fae8564d not found: ID does not exist"
Nov 04 03:19:10.845789 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.845749    4420 scope.go:122] "RemoveContainer" containerID="451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"
Nov 04 03:19:10.845967 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.845952    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce"} err="failed to get container status \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": rpc error: code = NotFound desc = could not find container \"451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce\": container with ID starting with 451585c9a732773b8ad1a94e35ae0b74ecdb16287b591369d183c5e26bb806ce not found: ID does not exist"
Nov 04 03:19:10.846019 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.845967    4420 scope.go:122] "RemoveContainer" containerID="187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"
Nov 04 03:19:10.846173 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.846156    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396"} err="failed to get container status \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": rpc error: code = NotFound desc = could not find container \"187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396\": container with ID starting with 187d6a157ea4ce9c2587997f5ccf005db32821cb7b4177f27fe562410cf42396 not found: ID does not exist"
Nov 04 03:19:10.846214 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.846173    4420 scope.go:122] "RemoveContainer" containerID="4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"
Nov 04 03:19:10.846377 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.846363    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725"} err="failed to get container status \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": rpc error: code = NotFound desc = could not find container \"4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725\": container with ID starting with 4c5a951b7e9f912a31e43b75d819834afe5bbd0e04a792286968e68bd08ee725 not found: ID does not exist"
Nov 04 03:19:10.846415 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.846377    4420 scope.go:122] "RemoveContainer" containerID="57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"
Nov 04 03:19:10.846569 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.846553    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc"} err="failed to get container status \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": rpc error: code = NotFound desc = could not find container \"57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc\": container with ID starting with 57b17124cbd28bbe9f6c0e05b01bf40389d523285a13a352e55a10c0cff001cc not found: ID does not exist"
Nov 04 03:19:10.951050 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.951027    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 03:19:10.997616 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.997591    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 03:19:10.998263 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998240    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 03:19:10.998485 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998473    4420 cpu_manager.go:414] "containerMap: removing container" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="kube-rbac-proxy"
Nov 04 03:19:10.998539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998486    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="kube-rbac-proxy"
Nov 04 03:19:10.998539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998500    4420 cpu_manager.go:414] "containerMap: removing container" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="init-config-reloader"
Nov 04 03:19:10.998539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998505    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="init-config-reloader"
Nov 04 03:19:10.998539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998511    4420 cpu_manager.go:414] "containerMap: removing container" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="thanos-sidecar"
Nov 04 03:19:10.998539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998516    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="thanos-sidecar"
Nov 04 03:19:10.998539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998521    4420 cpu_manager.go:414] "containerMap: removing container" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="kube-rbac-proxy-thanos"
Nov 04 03:19:10.998539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998526    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="kube-rbac-proxy-thanos"
Nov 04 03:19:10.998539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998535    4420 cpu_manager.go:414] "containerMap: removing container" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="prometheus"
Nov 04 03:19:10.998539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998539    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="prometheus"
Nov 04 03:19:10.998782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998554    4420 cpu_manager.go:414] "containerMap: removing container" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="kube-rbac-proxy-web"
Nov 04 03:19:10.998782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998558    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="kube-rbac-proxy-web"
Nov 04 03:19:10.998782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998566    4420 cpu_manager.go:414] "containerMap: removing container" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="config-reloader"
Nov 04 03:19:10.998782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998570    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="config-reloader"
Nov 04 03:19:10.998782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998610    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="config-reloader"
Nov 04 03:19:10.998782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998617    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="thanos-sidecar"
Nov 04 03:19:10.998782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998622    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="kube-rbac-proxy"
Nov 04 03:19:10.998782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998628    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="kube-rbac-proxy-thanos"
Nov 04 03:19:10.998782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998634    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="prometheus"
Nov 04 03:19:10.998782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:10.998638    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="667f8d76-bee3-4b64-afb7-59c789573285" containerName="kube-rbac-proxy-web"
Nov 04 03:19:11.002269 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.002252    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.052328 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052268    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 04 03:19:11.052328 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052295    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 04 03:19:11.052328 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052320    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 04 03:19:11.052491 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052338    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 04 03:19:11.052674 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052602    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 04 03:19:11.052674 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052616    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 04 03:19:11.052674 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052619    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 03:19:11.052674 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052635    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 04 03:19:11.052674 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052656    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 04 03:19:11.052674 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052676    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-15iq5iq7eocur\""
Nov 04 03:19:11.053124 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052684    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 04 03:19:11.053124 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052708    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 04 03:19:11.053124 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052795    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 04 03:19:11.053124 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052819    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kr76c\""
Nov 04 03:19:11.053124 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.052894    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 04 03:19:11.083159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083132    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-web-config\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083258 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083169    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083258 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083198    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21860859-c6cc-4399-80db-f7da70e90cc3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083334 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083301    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083334 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083329    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083419 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083349    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083419 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083398    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21860859-c6cc-4399-80db-f7da70e90cc3-config-out\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083512 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083493    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083555 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083541    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083565    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083585    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqf5p\" (UniqueName: \"kubernetes.io/projected/21860859-c6cc-4399-80db-f7da70e90cc3-kube-api-access-nqf5p\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083668 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083625    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-config\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083668 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083642    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083668 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083658    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083811 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083679    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083811 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083699    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083811 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083798    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083956 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083829    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083956 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083880    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.083956 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.083906    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.100144 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.100125    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 04 03:19:11.100283 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.100267    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 04 03:19:11.119837 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.119817    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 03:19:11.184798 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.184768    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.184935 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.184809    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.184935 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.184840    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.184935 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.184899    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21860859-c6cc-4399-80db-f7da70e90cc3-config-out\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.184935 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.184927    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185135 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.184957    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185135 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.184982    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185135 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185009    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqf5p\" (UniqueName: \"kubernetes.io/projected/21860859-c6cc-4399-80db-f7da70e90cc3-kube-api-access-nqf5p\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185135 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185040    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-config\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185135 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185064    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185135 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185090    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185135 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185125    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185484 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185159    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185484 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185195    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185484 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185221    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185484 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185244    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185484 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185278    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185484 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185313    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-web-config\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185484 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185345    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185484 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185371    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21860859-c6cc-4399-80db-f7da70e90cc3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185958 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185506    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.185958 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.185614    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.187353 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.186192    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.187353 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.186387    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.187353 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.187041    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.187353 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.187335    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.188526 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.188501    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.188769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.188746    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.189336 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.189287    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/21860859-c6cc-4399-80db-f7da70e90cc3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.190174 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.190024    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21860859-c6cc-4399-80db-f7da70e90cc3-config-out\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.190174 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.190152    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-web-config\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.190325 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.190284    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.190428 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.190406    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.190474 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.190410    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.190554 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.190533    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21860859-c6cc-4399-80db-f7da70e90cc3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.190610 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.190580    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.190924 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.190901    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.191004 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.190924    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.191650 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.191631    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/21860859-c6cc-4399-80db-f7da70e90cc3-config\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.244968 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.244942    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqf5p\" (UniqueName: \"kubernetes.io/projected/21860859-c6cc-4399-80db-f7da70e90cc3-kube-api-access-nqf5p\") pod \"prometheus-k8s-0\" (UID: \"21860859-c6cc-4399-80db-f7da70e90cc3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.319312 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.319259    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:11.746549 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.746518    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 03:19:11.753205 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:19:11.753182    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21860859_c6cc_4399_80db_f7da70e90cc3.slice/crio-4b547c625d3ec6428f02457c2c23757ea93ca97005ba00e62762c15bddd2158a WatchSource:0}: Error finding container 4b547c625d3ec6428f02457c2c23757ea93ca97005ba00e62762c15bddd2158a: Status 404 returned error can't find the container with id 4b547c625d3ec6428f02457c2c23757ea93ca97005ba00e62762c15bddd2158a
Nov 04 03:19:11.795652 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:11.795627    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21860859-c6cc-4399-80db-f7da70e90cc3","Type":"ContainerStarted","Data":"4b547c625d3ec6428f02457c2c23757ea93ca97005ba00e62762c15bddd2158a"}
Nov 04 03:19:12.308659 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:12.308628    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="667f8d76-bee3-4b64-afb7-59c789573285" path="/var/lib/kubelet/pods/667f8d76-bee3-4b64-afb7-59c789573285/volumes"
Nov 04 03:19:12.368133 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:12.368104    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:12.517255 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:12.517224    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:12.760175 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:12.760146    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:12.801641 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:12.801608    4420 generic.go:356] "Generic (PLEG): container finished" podID="21860859-c6cc-4399-80db-f7da70e90cc3" containerID="e92db380e308648cb6867259e334da70a81514cdb8d3cb42f9b01df3c41ed9cd" exitCode=0
Nov 04 03:19:12.801790 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:12.801645    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21860859-c6cc-4399-80db-f7da70e90cc3","Type":"ContainerDied","Data":"e92db380e308648cb6867259e334da70a81514cdb8d3cb42f9b01df3c41ed9cd"}
Nov 04 03:19:12.910543 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:12.910495    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:13.807661 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:13.807636    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:19:13.808082 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:13.807968    4420 generic.go:356] "Generic (PLEG): container finished" podID="21860859-c6cc-4399-80db-f7da70e90cc3" containerID="0c160ed83dee080806e1c829e5f9644fd4b56656e4be7604943227574f682f40" exitCode=2
Nov 04 03:19:13.808082 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:13.808002    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21860859-c6cc-4399-80db-f7da70e90cc3","Type":"ContainerStarted","Data":"40296732901c6bc1bca7406ad9a9d561827acc70994d2651730eaaeab64690dd"}
Nov 04 03:19:13.808082 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:13.808063    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21860859-c6cc-4399-80db-f7da70e90cc3","Type":"ContainerStarted","Data":"56699d4b97ee604bba24027f5ab19ebe1186c042360f71f1f02cd0518595f979"}
Nov 04 03:19:13.808199 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:13.808100    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21860859-c6cc-4399-80db-f7da70e90cc3","Type":"ContainerStarted","Data":"9a23bf0aa006b9553630f91ec6025883717fc726b1fe4600c597819d5406f1dc"}
Nov 04 03:19:13.808199 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:13.808137    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21860859-c6cc-4399-80db-f7da70e90cc3","Type":"ContainerStarted","Data":"c7e5f60ff1922832c0ae17084a6089d302c9708266d1e727d98fe24eacc3e39e"}
Nov 04 03:19:13.808199 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:13.808177    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21860859-c6cc-4399-80db-f7da70e90cc3","Type":"ContainerStarted","Data":"bc47f492e830a230144646ac916189c8857e8696e8c2344569e4eb508f94d29a"}
Nov 04 03:19:13.808283 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:13.808212    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21860859-c6cc-4399-80db-f7da70e90cc3","Type":"ContainerDied","Data":"0c160ed83dee080806e1c829e5f9644fd4b56656e4be7604943227574f682f40"}
Nov 04 03:19:13.808456 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:13.808442    4420 scope.go:122] "RemoveContainer" containerID="0c160ed83dee080806e1c829e5f9644fd4b56656e4be7604943227574f682f40"
Nov 04 03:19:14.814523 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:14.814496    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:19:14.814972 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:14.814906    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21860859-c6cc-4399-80db-f7da70e90cc3","Type":"ContainerStarted","Data":"b44c65f833f87e21ee86bc322a7a708df929a5309a0d60ee2151df70f6a6cff2"}
Nov 04 03:19:14.988083 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:14.988032    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.967885298 podStartE2EDuration="4.988017331s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.132033ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:19:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:19:14.987211075 +0000 UTC m=+124.802002352" watchObservedRunningTime="2026-11-04 03:19:14.988017331 +0000 UTC m=+124.802808662"
Nov 04 03:19:16.320292 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:16.320254    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:19:32.647294 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:19:32.647254    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:19:58.069638 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:58.069602    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:19:58.270204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:19:58.270167    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:00.631791 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:00.631749    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:02.647293 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:20:02.647258    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:20:04.005555 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:04.005519    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:11.319790 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:11.319754    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:20:11.334653 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:11.334624    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:20:12.026826 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:12.026792    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 03:20:19.761660 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:19.761619    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:26.424167 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:26.424130    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:26.622090 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:26.622032    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:26.814037 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:26.813999    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:26.958695 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:26.958656    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:32.594456 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:32.594400    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:32.648715 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:20:32.648678    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:20:32.789109 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:32.789076    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:46.206445 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:46.206385    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:20:46.355251 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:20:46.355219    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:21:00.989824 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:21:00.989721    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:21:02.648700 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:21:02.648665    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:21:32.648046 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:21:32.648009    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:22:01.331885 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:22:01.331823    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:22:02.647720 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:22:02.647679    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:22:10.300512 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:22:10.300483    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:22:10.307672 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:22:10.307647    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:22:32.647492 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:22:32.647448    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:22:50.230561 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:22:50.230522    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:22:50.403079 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:22:50.403042    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:22:51.778348 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:22:51.778300    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:22:51.964567 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:22:51.964530    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:22:53.091572 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:22:53.091528    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:22:58.285250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:22:58.285209    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:22:58.498013 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:22:58.497973    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:22:59.707850 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:22:59.707805    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:22:59.871739 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:22:59.871706    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:00.023954 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:00.023917    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:00.170480 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:00.170439    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:01.681490 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:01.681454    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:02.649614 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:23:02.649579    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:23:03.937600 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:03.937562    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:32.648280 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:23:32.648241    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:23:38.910302 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:38.910258    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:39.104293 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:39.104257    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:45.066466 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:45.066427    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:45.285547 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:45.285515    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:46.290119 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.290080    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:46.577979 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.577896    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"]
Nov 04 03:23:46.580394 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.580377    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.640915 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.640882    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 03:23:46.641104 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.640966    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 03:23:46.641104 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.641067    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-9kg9c\""
Nov 04 03:23:46.674512 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.674475    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d056a780-6e7f-463e-a706-26938ec7fbf1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.674695 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.674522    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d056a780-6e7f-463e-a706-26938ec7fbf1-manifests\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.674695 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.674605    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d056a780-6e7f-463e-a706-26938ec7fbf1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.674695 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.674626    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqt7p\" (UniqueName: \"kubernetes.io/projected/d056a780-6e7f-463e-a706-26938ec7fbf1-kube-api-access-jqt7p\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.674903 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.674728    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d056a780-6e7f-463e-a706-26938ec7fbf1-tmp\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.674903 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.674792    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d056a780-6e7f-463e-a706-26938ec7fbf1-metrics-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.691707 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.687565    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 03:23:46.693164 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.693143    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"]
Nov 04 03:23:46.713143 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.713117    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:46.734242 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.734207    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 03:23:46.734242 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.734230    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 03:23:46.775930 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.775893    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d056a780-6e7f-463e-a706-26938ec7fbf1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.775930 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.775930    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqt7p\" (UniqueName: \"kubernetes.io/projected/d056a780-6e7f-463e-a706-26938ec7fbf1-kube-api-access-jqt7p\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.776130 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.775959    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d056a780-6e7f-463e-a706-26938ec7fbf1-tmp\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.776130 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.776006    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d056a780-6e7f-463e-a706-26938ec7fbf1-metrics-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.776130 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.776027    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d056a780-6e7f-463e-a706-26938ec7fbf1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.776267 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.776189    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d056a780-6e7f-463e-a706-26938ec7fbf1-manifests\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.776553 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.776517    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d056a780-6e7f-463e-a706-26938ec7fbf1-manifests\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.778351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.778326    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d056a780-6e7f-463e-a706-26938ec7fbf1-tmp\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.778453 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.778422    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d056a780-6e7f-463e-a706-26938ec7fbf1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.778605 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.778580    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d056a780-6e7f-463e-a706-26938ec7fbf1-metrics-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.778665 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.778614    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d056a780-6e7f-463e-a706-26938ec7fbf1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.836697 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.836622    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqt7p\" (UniqueName: \"kubernetes.io/projected/d056a780-6e7f-463e-a706-26938ec7fbf1-kube-api-access-jqt7p\") pod \"opendatahub-operator-controller-manager-6ffdd4467b-f8hl6\" (UID: \"d056a780-6e7f-463e-a706-26938ec7fbf1\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:46.866560 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.866517    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:46.894204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:46.894169    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:47.017001 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:47.016966    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:47.201927 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:47.201830    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:47.353686 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:47.353648    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:47.380634 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:47.380597    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"]
Nov 04 03:23:47.387908 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:23:47.387879    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd056a780_6e7f_463e_a706_26938ec7fbf1.slice/crio-fe661106970f44d169ddde9404a926dcf753e9c28c1b262d3f87485e352a677b WatchSource:0}: Error finding container fe661106970f44d169ddde9404a926dcf753e9c28c1b262d3f87485e352a677b: Status 404 returned error can't find the container with id fe661106970f44d169ddde9404a926dcf753e9c28c1b262d3f87485e352a677b
Nov 04 03:23:47.389602 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:47.389583    4420 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:23:47.724460 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:47.724427    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6" event={"ID":"d056a780-6e7f-463e-a706-26938ec7fbf1","Type":"ContainerStarted","Data":"fe661106970f44d169ddde9404a926dcf753e9c28c1b262d3f87485e352a677b"}
Nov 04 03:23:48.840695 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:48.840655    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:50.580383 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:50.580333    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:51.492972 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:51.492877    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:51.695504 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:51.695477    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:51.742393 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:51.742358    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6" event={"ID":"d056a780-6e7f-463e-a706-26938ec7fbf1","Type":"ContainerStarted","Data":"f7d8af885248ac436e89f4195790d38ae69735cef01eaaafd6b09690878d1ff5"}
Nov 04 03:23:51.915118 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:51.915026    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:52.747835 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:52.747800    4420 generic.go:356] "Generic (PLEG): container finished" podID="d056a780-6e7f-463e-a706-26938ec7fbf1" containerID="f7d8af885248ac436e89f4195790d38ae69735cef01eaaafd6b09690878d1ff5" exitCode=0
Nov 04 03:23:52.748231 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:52.747886    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6" event={"ID":"d056a780-6e7f-463e-a706-26938ec7fbf1","Type":"ContainerDied","Data":"f7d8af885248ac436e89f4195790d38ae69735cef01eaaafd6b09690878d1ff5"}
Nov 04 03:23:53.450907 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:53.450870    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:53.627609 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:53.627578    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:53.754197 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:53.754142    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6" event={"ID":"d056a780-6e7f-463e-a706-26938ec7fbf1","Type":"ContainerStarted","Data":"25b0cac94b079f372e995c7303e1342722992e1536c78308b7b93f029b57de10"}
Nov 04 03:23:53.754613 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:53.754228    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:23:53.887250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:53.887200    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6" podStartSLOduration=2.267199983 podStartE2EDuration="7.887183603s" totalImagesPullingTime="5.149980481s" totalInitContainerRuntime="470.003139ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:23:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:23:53.885956948 +0000 UTC m=+403.700748226" watchObservedRunningTime="2026-11-04 03:23:53.887183603 +0000 UTC m=+403.701974880"
Nov 04 03:23:57.075696 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:57.075655    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:57.233157 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:57.233107    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:57.396448 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:57.396358    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:57.563888 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:57.563854    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:23:59.570363 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:23:59.570323    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:00.283224 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.283193    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:00.355649 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.355620    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn"]
Nov 04 03:24:00.362799 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.362778    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn"
Nov 04 03:24:00.388708 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.388684    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5477e7bf-98d0-42cf-be2d-3e9a54a8f915-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-x67vn\" (UID: \"5477e7bf-98d0-42cf-be2d-3e9a54a8f915\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn"
Nov 04 03:24:00.388817 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.388713    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4n29\" (UniqueName: \"kubernetes.io/projected/5477e7bf-98d0-42cf-be2d-3e9a54a8f915-kube-api-access-b4n29\") pod \"cert-manager-cainjector-5c57bbd4fc-x67vn\" (UID: \"5477e7bf-98d0-42cf-be2d-3e9a54a8f915\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn"
Nov 04 03:24:00.413654 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.413628    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn"]
Nov 04 03:24:00.459248 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.459228    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 04 03:24:00.459390 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.459252    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 04 03:24:00.459390 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.459260    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-24xmf\""
Nov 04 03:24:00.490069 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.490037    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5477e7bf-98d0-42cf-be2d-3e9a54a8f915-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-x67vn\" (UID: \"5477e7bf-98d0-42cf-be2d-3e9a54a8f915\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn"
Nov 04 03:24:00.490203 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.490074    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4n29\" (UniqueName: \"kubernetes.io/projected/5477e7bf-98d0-42cf-be2d-3e9a54a8f915-kube-api-access-b4n29\") pod \"cert-manager-cainjector-5c57bbd4fc-x67vn\" (UID: \"5477e7bf-98d0-42cf-be2d-3e9a54a8f915\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn"
Nov 04 03:24:00.552284 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.552210    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5477e7bf-98d0-42cf-be2d-3e9a54a8f915-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-x67vn\" (UID: \"5477e7bf-98d0-42cf-be2d-3e9a54a8f915\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn"
Nov 04 03:24:00.553168 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.553151    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4n29\" (UniqueName: \"kubernetes.io/projected/5477e7bf-98d0-42cf-be2d-3e9a54a8f915-kube-api-access-b4n29\") pod \"cert-manager-cainjector-5c57bbd4fc-x67vn\" (UID: \"5477e7bf-98d0-42cf-be2d-3e9a54a8f915\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn"
Nov 04 03:24:00.684166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:00.684135    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn"
Nov 04 03:24:01.084521 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:01.084486    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn"]
Nov 04 03:24:01.091880 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:24:01.091834    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5477e7bf_98d0_42cf_be2d_3e9a54a8f915.slice/crio-50da0e8ae5d876908f2fda445b1f2d7be558a1a753ce2ba350b9cfcee7a037b1 WatchSource:0}: Error finding container 50da0e8ae5d876908f2fda445b1f2d7be558a1a753ce2ba350b9cfcee7a037b1: Status 404 returned error can't find the container with id 50da0e8ae5d876908f2fda445b1f2d7be558a1a753ce2ba350b9cfcee7a037b1
Nov 04 03:24:01.427457 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:01.427384    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:01.785089 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:01.785058    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn" event={"ID":"5477e7bf-98d0-42cf-be2d-3e9a54a8f915","Type":"ContainerStarted","Data":"50da0e8ae5d876908f2fda445b1f2d7be558a1a753ce2ba350b9cfcee7a037b1"}
Nov 04 03:24:02.026948 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:02.026920    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:02.437409 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:02.437376    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:02.649684 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:24:02.649647    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:24:02.856689 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:02.856660    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:03.053941 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:03.053901    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:03.230903 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:03.230791    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:03.956711 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:03.956671    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"]
Nov 04 03:24:03.960152 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:03.960125    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"
Nov 04 03:24:04.027021 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:04.026800    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18a06ece-6ffc-4c88-8261-38f3645e3b80-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-zsl4r\" (UID: \"18a06ece-6ffc-4c88-8261-38f3645e3b80\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"
Nov 04 03:24:04.027021 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:04.026931    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhvmq\" (UniqueName: \"kubernetes.io/projected/18a06ece-6ffc-4c88-8261-38f3645e3b80-kube-api-access-bhvmq\") pod \"cert-manager-webhook-5dbbbf75cc-zsl4r\" (UID: \"18a06ece-6ffc-4c88-8261-38f3645e3b80\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"
Nov 04 03:24:04.057093 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:04.057059    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bf42g\""
Nov 04 03:24:04.058275 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:04.058211    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"]
Nov 04 03:24:04.128247 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:04.128210    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhvmq\" (UniqueName: \"kubernetes.io/projected/18a06ece-6ffc-4c88-8261-38f3645e3b80-kube-api-access-bhvmq\") pod \"cert-manager-webhook-5dbbbf75cc-zsl4r\" (UID: \"18a06ece-6ffc-4c88-8261-38f3645e3b80\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"
Nov 04 03:24:04.128247 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:04.128256    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18a06ece-6ffc-4c88-8261-38f3645e3b80-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-zsl4r\" (UID: \"18a06ece-6ffc-4c88-8261-38f3645e3b80\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"
Nov 04 03:24:04.195277 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:04.195247    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18a06ece-6ffc-4c88-8261-38f3645e3b80-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-zsl4r\" (UID: \"18a06ece-6ffc-4c88-8261-38f3645e3b80\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"
Nov 04 03:24:04.200351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:04.200289    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhvmq\" (UniqueName: \"kubernetes.io/projected/18a06ece-6ffc-4c88-8261-38f3645e3b80-kube-api-access-bhvmq\") pod \"cert-manager-webhook-5dbbbf75cc-zsl4r\" (UID: \"18a06ece-6ffc-4c88-8261-38f3645e3b80\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"
Nov 04 03:24:04.274926 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:04.274663    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"
Nov 04 03:24:04.664935 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:04.664908    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"]
Nov 04 03:24:04.672781 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:24:04.672750    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18a06ece_6ffc_4c88_8261_38f3645e3b80.slice/crio-5752002e756c114ffc95b5d14bc603c9d31ca6b6f0808d716b09fdfebc031d9b WatchSource:0}: Error finding container 5752002e756c114ffc95b5d14bc603c9d31ca6b6f0808d716b09fdfebc031d9b: Status 404 returned error can't find the container with id 5752002e756c114ffc95b5d14bc603c9d31ca6b6f0808d716b09fdfebc031d9b
Nov 04 03:24:04.760055 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:04.760021    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6ffdd4467b-f8hl6"
Nov 04 03:24:04.799311 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:04.799247    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r" event={"ID":"18a06ece-6ffc-4c88-8261-38f3645e3b80","Type":"ContainerStarted","Data":"5752002e756c114ffc95b5d14bc603c9d31ca6b6f0808d716b09fdfebc031d9b"}
Nov 04 03:24:05.078479 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:05.078440    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:06.809873 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:06.809824    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn" event={"ID":"5477e7bf-98d0-42cf-be2d-3e9a54a8f915","Type":"ContainerStarted","Data":"7759cd6783ca624981ef5bf7da84f1ef49a1ec4cf6f266a7ffa292cd364e2ed7"}
Nov 04 03:24:06.920168 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:06.920116    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-x67vn" podStartSLOduration=1.741774435 podStartE2EDuration="6.920102686s" totalImagesPullingTime="5.178328251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:06.919264745 +0000 UTC m=+416.734056032" watchObservedRunningTime="2026-11-04 03:24:06.920102686 +0000 UTC m=+416.734893962"
Nov 04 03:24:07.815496 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:07.815462    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r" event={"ID":"18a06ece-6ffc-4c88-8261-38f3645e3b80","Type":"ContainerStarted","Data":"2e305fe13340d2bf6963d6767adc509aaef8078cd6e32c274e9f113edb29c6ce"}
Nov 04 03:24:07.943737 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:07.943634    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r" podStartSLOduration=1.99943654 podStartE2EDuration="4.943618176s" totalImagesPullingTime="2.944181636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:07.942102451 +0000 UTC m=+417.756893728" watchObservedRunningTime="2026-11-04 03:24:07.943618176 +0000 UTC m=+417.758409452"
Nov 04 03:24:07.970772 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:07.970731    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:08.196775 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:08.196681    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:08.345767 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:08.345733    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:08.500450 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:08.500411    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:08.818678 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:08.818594    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"
Nov 04 03:24:09.270196 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:09.270160    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:10.708353 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:10.708320    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:13.060452 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:13.060413    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:13.296451 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:13.296419    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:14.816081 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:14.816046    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt"]
Nov 04 03:24:14.820637 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:14.820614    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt"
Nov 04 03:24:14.824530 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:14.824508    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-zsl4r"
Nov 04 03:24:14.922511 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:14.922476    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 03:24:14.922691 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:14.922638    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jnd9j\""
Nov 04 03:24:14.922691 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:14.922653    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 03:24:14.923751 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:14.923731    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt"]
Nov 04 03:24:14.926110 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:14.926094    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4p97\" (UniqueName: \"kubernetes.io/projected/ce464396-b997-492c-865c-5def7527f977-kube-api-access-p4p97\") pod \"limitador-operator-controller-manager-5778bb4449-jtvzt\" (UID: \"ce464396-b997-492c-865c-5def7527f977\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt"
Nov 04 03:24:15.027092 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:15.027047    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4p97\" (UniqueName: \"kubernetes.io/projected/ce464396-b997-492c-865c-5def7527f977-kube-api-access-p4p97\") pod \"limitador-operator-controller-manager-5778bb4449-jtvzt\" (UID: \"ce464396-b997-492c-865c-5def7527f977\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt"
Nov 04 03:24:15.097724 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:15.097655    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4p97\" (UniqueName: \"kubernetes.io/projected/ce464396-b997-492c-865c-5def7527f977-kube-api-access-p4p97\") pod \"limitador-operator-controller-manager-5778bb4449-jtvzt\" (UID: \"ce464396-b997-492c-865c-5def7527f977\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt"
Nov 04 03:24:15.133320 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:15.133268    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt"
Nov 04 03:24:15.517239 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:15.517210    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt"]
Nov 04 03:24:15.524136 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:24:15.524111    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce464396_b997_492c_865c_5def7527f977.slice/crio-819e8b5f0995dd0ab05f2e401264496261a98d49152e22036a1d9fe82809f787 WatchSource:0}: Error finding container 819e8b5f0995dd0ab05f2e401264496261a98d49152e22036a1d9fe82809f787: Status 404 returned error can't find the container with id 819e8b5f0995dd0ab05f2e401264496261a98d49152e22036a1d9fe82809f787
Nov 04 03:24:15.843711 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:15.843624    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt" event={"ID":"ce464396-b997-492c-865c-5def7527f977","Type":"ContainerStarted","Data":"819e8b5f0995dd0ab05f2e401264496261a98d49152e22036a1d9fe82809f787"}
Nov 04 03:24:16.046826 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:16.046790    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:16.205190 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:16.205116    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:17.946415 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:17.946364    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-d2kl5"]
Nov 04 03:24:17.950306 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:17.949942    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-d2kl5"
Nov 04 03:24:18.045659 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.045585    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-2px5r\""
Nov 04 03:24:18.046884 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.046862    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-d2kl5"]
Nov 04 03:24:18.050788 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.050759    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tqls\" (UniqueName: \"kubernetes.io/projected/5422d24b-a673-4ac1-960a-9a39f0f176c4-kube-api-access-7tqls\") pod \"cert-manager-597b95cb84-d2kl5\" (UID: \"5422d24b-a673-4ac1-960a-9a39f0f176c4\") " pod="cert-manager/cert-manager-597b95cb84-d2kl5"
Nov 04 03:24:18.050914 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.050810    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5422d24b-a673-4ac1-960a-9a39f0f176c4-bound-sa-token\") pod \"cert-manager-597b95cb84-d2kl5\" (UID: \"5422d24b-a673-4ac1-960a-9a39f0f176c4\") " pod="cert-manager/cert-manager-597b95cb84-d2kl5"
Nov 04 03:24:18.152714 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.152557    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tqls\" (UniqueName: \"kubernetes.io/projected/5422d24b-a673-4ac1-960a-9a39f0f176c4-kube-api-access-7tqls\") pod \"cert-manager-597b95cb84-d2kl5\" (UID: \"5422d24b-a673-4ac1-960a-9a39f0f176c4\") " pod="cert-manager/cert-manager-597b95cb84-d2kl5"
Nov 04 03:24:18.152714 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.152611    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5422d24b-a673-4ac1-960a-9a39f0f176c4-bound-sa-token\") pod \"cert-manager-597b95cb84-d2kl5\" (UID: \"5422d24b-a673-4ac1-960a-9a39f0f176c4\") " pod="cert-manager/cert-manager-597b95cb84-d2kl5"
Nov 04 03:24:18.216815 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.216741    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tqls\" (UniqueName: \"kubernetes.io/projected/5422d24b-a673-4ac1-960a-9a39f0f176c4-kube-api-access-7tqls\") pod \"cert-manager-597b95cb84-d2kl5\" (UID: \"5422d24b-a673-4ac1-960a-9a39f0f176c4\") " pod="cert-manager/cert-manager-597b95cb84-d2kl5"
Nov 04 03:24:18.217776 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.217755    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5422d24b-a673-4ac1-960a-9a39f0f176c4-bound-sa-token\") pod \"cert-manager-597b95cb84-d2kl5\" (UID: \"5422d24b-a673-4ac1-960a-9a39f0f176c4\") " pod="cert-manager/cert-manager-597b95cb84-d2kl5"
Nov 04 03:24:18.268163 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.268127    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-d2kl5"
Nov 04 03:24:18.641221 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.641185    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-d2kl5"]
Nov 04 03:24:18.648982 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:24:18.648954    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5422d24b_a673_4ac1_960a_9a39f0f176c4.slice/crio-c7c108cbf1261fa2ddcf4ceabc0fd4fa09e16400a6b39e6dc7e5b00daa109a74 WatchSource:0}: Error finding container c7c108cbf1261fa2ddcf4ceabc0fd4fa09e16400a6b39e6dc7e5b00daa109a74: Status 404 returned error can't find the container with id c7c108cbf1261fa2ddcf4ceabc0fd4fa09e16400a6b39e6dc7e5b00daa109a74
Nov 04 03:24:18.858794 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.858760    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-d2kl5" event={"ID":"5422d24b-a673-4ac1-960a-9a39f0f176c4","Type":"ContainerStarted","Data":"7395b1f99449505633bd090ccf1633955fca2bb390444b7b8598eb2548fcedb5"}
Nov 04 03:24:18.858970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.858804    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-d2kl5" event={"ID":"5422d24b-a673-4ac1-960a-9a39f0f176c4","Type":"ContainerStarted","Data":"c7c108cbf1261fa2ddcf4ceabc0fd4fa09e16400a6b39e6dc7e5b00daa109a74"}
Nov 04 03:24:18.974552 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:18.974467    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-d2kl5" podStartSLOduration=1.974453808 podStartE2EDuration="1.974453808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:18.973656967 +0000 UTC m=+428.788448248" watchObservedRunningTime="2026-11-04 03:24:18.974453808 +0000 UTC m=+428.789245085"
Nov 04 03:24:19.864540 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:19.864448    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt" event={"ID":"ce464396-b997-492c-865c-5def7527f977","Type":"ContainerStarted","Data":"f5594f8af455a30696a3a546f7e79a84110ca4376df5f06bdf884085a4d37d83"}
Nov 04 03:24:19.864540 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:19.864521    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt"
Nov 04 03:24:19.870876 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:19.870835    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:19.974874 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:19.974809    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt" podStartSLOduration=1.930247714 podStartE2EDuration="5.974795257s" totalImagesPullingTime="4.044547543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:19.9733255 +0000 UTC m=+429.788116778" watchObservedRunningTime="2026-11-04 03:24:19.974795257 +0000 UTC m=+429.789586533"
Nov 04 03:24:20.076095 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:20.076053    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:21.432178 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:21.432138    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:21.631404 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:21.631366    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:21.783755 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:21.783723    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:21.939329 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:21.939295    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:29.604421 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:29.604387    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:29.757414 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:29.757376    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:30.869858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:30.869810    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jtvzt"
Nov 04 03:24:32.508501 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:32.508465    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:32.648776 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:24:32.648740    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:24:32.681179 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:32.681149    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:32.888147 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:32.888075    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:33.044215 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:33.044167    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:33.535533 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:33.535496    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:35.133182 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.133127    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:35.278310 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.278272    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"]
Nov 04 03:24:35.284425 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.284393    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"
Nov 04 03:24:35.332757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.332726    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:35.340092 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.340058    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"]
Nov 04 03:24:35.385431 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.385355    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 03:24:35.385431 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.385354    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 04 03:24:35.385664 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.385432    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 04 03:24:35.385664 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.385447    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 03:24:35.385664 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.385478    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sgt86\""
Nov 04 03:24:35.391794 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.391767    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71822c20-2453-4e37-96a9-a0f20d9c11b8-tmp\") pod \"kube-auth-proxy-7659c7d778-lxgpd\" (UID: \"71822c20-2453-4e37-96a9-a0f20d9c11b8\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"
Nov 04 03:24:35.391948 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.391799    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh2kb\" (UniqueName: \"kubernetes.io/projected/71822c20-2453-4e37-96a9-a0f20d9c11b8-kube-api-access-zh2kb\") pod \"kube-auth-proxy-7659c7d778-lxgpd\" (UID: \"71822c20-2453-4e37-96a9-a0f20d9c11b8\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"
Nov 04 03:24:35.391948 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.391898    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71822c20-2453-4e37-96a9-a0f20d9c11b8-tls-certs\") pod \"kube-auth-proxy-7659c7d778-lxgpd\" (UID: \"71822c20-2453-4e37-96a9-a0f20d9c11b8\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"
Nov 04 03:24:35.492497 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.492460    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71822c20-2453-4e37-96a9-a0f20d9c11b8-tls-certs\") pod \"kube-auth-proxy-7659c7d778-lxgpd\" (UID: \"71822c20-2453-4e37-96a9-a0f20d9c11b8\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"
Nov 04 03:24:35.492693 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.492550    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71822c20-2453-4e37-96a9-a0f20d9c11b8-tmp\") pod \"kube-auth-proxy-7659c7d778-lxgpd\" (UID: \"71822c20-2453-4e37-96a9-a0f20d9c11b8\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"
Nov 04 03:24:35.492693 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.492576    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh2kb\" (UniqueName: \"kubernetes.io/projected/71822c20-2453-4e37-96a9-a0f20d9c11b8-kube-api-access-zh2kb\") pod \"kube-auth-proxy-7659c7d778-lxgpd\" (UID: \"71822c20-2453-4e37-96a9-a0f20d9c11b8\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"
Nov 04 03:24:35.494518 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.494473    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:35.495218 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.495188    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71822c20-2453-4e37-96a9-a0f20d9c11b8-tmp\") pod \"kube-auth-proxy-7659c7d778-lxgpd\" (UID: \"71822c20-2453-4e37-96a9-a0f20d9c11b8\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"
Nov 04 03:24:35.495347 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.495307    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71822c20-2453-4e37-96a9-a0f20d9c11b8-tls-certs\") pod \"kube-auth-proxy-7659c7d778-lxgpd\" (UID: \"71822c20-2453-4e37-96a9-a0f20d9c11b8\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"
Nov 04 03:24:35.551559 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.551532    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh2kb\" (UniqueName: \"kubernetes.io/projected/71822c20-2453-4e37-96a9-a0f20d9c11b8-kube-api-access-zh2kb\") pod \"kube-auth-proxy-7659c7d778-lxgpd\" (UID: \"71822c20-2453-4e37-96a9-a0f20d9c11b8\") " pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"
Nov 04 03:24:35.595518 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.595496    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"
Nov 04 03:24:35.648112 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.648080    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:35.861482 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.861454    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:35.964318 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:35.964247    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd"]
Nov 04 03:24:35.971023 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:24:35.971001    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71822c20_2453_4e37_96a9_a0f20d9c11b8.slice/crio-ae8becdb2bb8736280b3f63bd76891b71e20b4fab3699e8b6c66a58c2269d70d WatchSource:0}: Error finding container ae8becdb2bb8736280b3f63bd76891b71e20b4fab3699e8b6c66a58c2269d70d: Status 404 returned error can't find the container with id ae8becdb2bb8736280b3f63bd76891b71e20b4fab3699e8b6c66a58c2269d70d
Nov 04 03:24:36.024957 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:36.024922    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:36.177724 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:36.177687    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:36.932901 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:36.932866    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd" event={"ID":"71822c20-2453-4e37-96a9-a0f20d9c11b8","Type":"ContainerStarted","Data":"ae8becdb2bb8736280b3f63bd76891b71e20b4fab3699e8b6c66a58c2269d70d"}
Nov 04 03:24:37.726132 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:37.726091    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:37.981373 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:37.981223    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:38.214073 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:38.214036    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:39.160308 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:39.160249    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:39.362759 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:39.362715    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:39.949321 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:39.949242    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd" event={"ID":"71822c20-2453-4e37-96a9-a0f20d9c11b8","Type":"ContainerStarted","Data":"b00fffeb1c823f2e18c661b5191699fc2b5d8750f2df786438a9a3fd24f42cd0"}
Nov 04 03:24:40.059067 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.059021    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7659c7d778-lxgpd" podStartSLOduration=1.526729848 podStartE2EDuration="5.059008889s" totalImagesPullingTime="3.532279041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:40.058403321 +0000 UTC m=+449.873194598" watchObservedRunningTime="2026-11-04 03:24:40.059008889 +0000 UTC m=+449.873800377"
Nov 04 03:24:40.519362 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.519324    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:40.673570 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.673526    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:40.864402 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.864325    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-2f296"]
Nov 04 03:24:40.891782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.891752    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:24:40.891946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.891902    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-2f296"
Nov 04 03:24:40.894503 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.894482    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"
Nov 04 03:24:40.922160 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.922138    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-2f296"]
Nov 04 03:24:40.930986 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.930961    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:40.933941 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.933918    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5805a14d-7dad-43be-9437-292b8982a631-config-file\") pod \"limitador-limitador-7c9b87d574-rqz2j\" (UID: \"5805a14d-7dad-43be-9437-292b8982a631\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"
Nov 04 03:24:40.934042 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.933952    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bz2t\" (UniqueName: \"kubernetes.io/projected/5805a14d-7dad-43be-9437-292b8982a631-kube-api-access-7bz2t\") pod \"limitador-limitador-7c9b87d574-rqz2j\" (UID: \"5805a14d-7dad-43be-9437-292b8982a631\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"
Nov 04 03:24:40.941955 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.941935    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-trsqp\""
Nov 04 03:24:40.967550 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.967526    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cfrc6\""
Nov 04 03:24:40.967550 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.967526    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 04 03:24:40.968738 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:40.968714    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:24:41.034793 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.034765    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvkml\" (UniqueName: \"kubernetes.io/projected/ae6d187a-3d9b-400a-a31a-70ccb0a0ccae-kube-api-access-gvkml\") pod \"authorino-56c6c9ddd5-2f296\" (UID: \"ae6d187a-3d9b-400a-a31a-70ccb0a0ccae\") " pod="kuadrant-system/authorino-56c6c9ddd5-2f296"
Nov 04 03:24:41.034793 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.034794    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5805a14d-7dad-43be-9437-292b8982a631-config-file\") pod \"limitador-limitador-7c9b87d574-rqz2j\" (UID: \"5805a14d-7dad-43be-9437-292b8982a631\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"
Nov 04 03:24:41.035006 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.034817    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bz2t\" (UniqueName: \"kubernetes.io/projected/5805a14d-7dad-43be-9437-292b8982a631-kube-api-access-7bz2t\") pod \"limitador-limitador-7c9b87d574-rqz2j\" (UID: \"5805a14d-7dad-43be-9437-292b8982a631\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"
Nov 04 03:24:41.035603 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.035584    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5805a14d-7dad-43be-9437-292b8982a631-config-file\") pod \"limitador-limitador-7c9b87d574-rqz2j\" (UID: \"5805a14d-7dad-43be-9437-292b8982a631\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"
Nov 04 03:24:41.080916 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.080882    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:41.092124 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.092097    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bz2t\" (UniqueName: \"kubernetes.io/projected/5805a14d-7dad-43be-9437-292b8982a631-kube-api-access-7bz2t\") pod \"limitador-limitador-7c9b87d574-rqz2j\" (UID: \"5805a14d-7dad-43be-9437-292b8982a631\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"
Nov 04 03:24:41.135314 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.135249    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvkml\" (UniqueName: \"kubernetes.io/projected/ae6d187a-3d9b-400a-a31a-70ccb0a0ccae-kube-api-access-gvkml\") pod \"authorino-56c6c9ddd5-2f296\" (UID: \"ae6d187a-3d9b-400a-a31a-70ccb0a0ccae\") " pod="kuadrant-system/authorino-56c6c9ddd5-2f296"
Nov 04 03:24:41.162330 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.162304    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:24:41.193353 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.193325    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvkml\" (UniqueName: \"kubernetes.io/projected/ae6d187a-3d9b-400a-a31a-70ccb0a0ccae-kube-api-access-gvkml\") pod \"authorino-56c6c9ddd5-2f296\" (UID: \"ae6d187a-3d9b-400a-a31a-70ccb0a0ccae\") " pod="kuadrant-system/authorino-56c6c9ddd5-2f296"
Nov 04 03:24:41.204158 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.204137    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-2f296"
Nov 04 03:24:41.210219 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.210200    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"
Nov 04 03:24:41.317913 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.317867    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:41.468877 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.468787    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:41.568975 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.568943    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-2f296"]
Nov 04 03:24:41.576503 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:24:41.576476    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae6d187a_3d9b_400a_a31a_70ccb0a0ccae.slice/crio-5da6c811667cf39dd4ad6791289fdcc52b2300d5570721ddc9b288fa18549e99 WatchSource:0}: Error finding container 5da6c811667cf39dd4ad6791289fdcc52b2300d5570721ddc9b288fa18549e99: Status 404 returned error can't find the container with id 5da6c811667cf39dd4ad6791289fdcc52b2300d5570721ddc9b288fa18549e99
Nov 04 03:24:41.620758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.620730    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:24:41.627445 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:24:41.627420    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5805a14d_7dad_43be_9437_292b8982a631.slice/crio-904342728fa444960ffc4429b1f80be2e5aa021e6ed0ca087fb1b373d64f8140 WatchSource:0}: Error finding container 904342728fa444960ffc4429b1f80be2e5aa021e6ed0ca087fb1b373d64f8140: Status 404 returned error can't find the container with id 904342728fa444960ffc4429b1f80be2e5aa021e6ed0ca087fb1b373d64f8140
Nov 04 03:24:41.959039 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.958999    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j" event={"ID":"5805a14d-7dad-43be-9437-292b8982a631","Type":"ContainerStarted","Data":"904342728fa444960ffc4429b1f80be2e5aa021e6ed0ca087fb1b373d64f8140"}
Nov 04 03:24:41.960191 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:41.960163    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-2f296" event={"ID":"ae6d187a-3d9b-400a-a31a-70ccb0a0ccae","Type":"ContainerStarted","Data":"5da6c811667cf39dd4ad6791289fdcc52b2300d5570721ddc9b288fa18549e99"}
Nov 04 03:24:42.734373 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:42.734335    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:44.972771 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:44.972739    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-2f296" event={"ID":"ae6d187a-3d9b-400a-a31a-70ccb0a0ccae","Type":"ContainerStarted","Data":"826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7"}
Nov 04 03:24:45.079891 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:45.079823    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-2f296" podStartSLOduration=2.418083287 podStartE2EDuration="5.079811199s" totalImagesPullingTime="2.661727912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:45.07944851 +0000 UTC m=+454.894239787" watchObservedRunningTime="2026-11-04 03:24:45.079811199 +0000 UTC m=+454.894602476"
Nov 04 03:24:49.329902 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:49.329834    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:49.535383 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:49.535348    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:52.319499 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:52.319461    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:52.470143 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:52.470106    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:24:55.014119 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:55.014081    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j" event={"ID":"5805a14d-7dad-43be-9437-292b8982a631","Type":"ContainerStarted","Data":"a46b4d2a386b1330419b550a124a79a99d8e411dc1fb9ee3ad89bbf951742319"}
Nov 04 03:24:55.014506 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:55.014211    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"
Nov 04 03:24:55.123726 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:24:55.123669    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j" podStartSLOduration=2.470088564 podStartE2EDuration="15.123652858s" totalImagesPullingTime="12.653564294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:24:55.122906023 +0000 UTC m=+464.937697296" watchObservedRunningTime="2026-11-04 03:24:55.123652858 +0000 UTC m=+464.938444135"
Nov 04 03:25:01.143325 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:01.143291    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:01.342584 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:01.342551    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:02.380098 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:02.380069    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:02.648824 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:25:02.648750    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:25:06.018970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:06.018940    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"
Nov 04 03:25:15.988343 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:15.988300    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:16.208754 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:16.208721    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:16.714695 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:16.714663    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:32.648364 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:25:32.648329    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:25:38.446856 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:38.446806    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:38.647163 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:38.647132    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:38.798785 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:38.798753    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:38.948420 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:38.948377    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:45.513332 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:45.513297    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:45.713125 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:45.713089    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:46.987274 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:46.987236    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:47.163500 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:47.163464    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:50.180370 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:50.180329    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:25:50.335357 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:25:50.335300    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:02.646939 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:02.646905    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:02.649387 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:26:02.649362    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:26:02.800705 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:02.800667    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:05.156860 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:05.156809    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:08.468456 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:08.468415    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:08.616502 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:08.616458    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:08.769182 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:08.769151    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:08.913745 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:08.913706    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:09.064949 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.064862    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:09.211684 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.211652    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:09.344039 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.343959    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-rcb44"]
Nov 04 03:26:09.347275 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.347253    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44"
Nov 04 03:26:09.359858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.359810    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:09.408114 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.408083    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 03:26:09.434296 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.434267    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkj8k\" (UniqueName: \"kubernetes.io/projected/2fe2560c-8be3-41e7-a487-70c9075c72ba-kube-api-access-gkj8k\") pod \"kserve-controller-manager-776cc66d47-rcb44\" (UID: \"2fe2560c-8be3-41e7-a487-70c9075c72ba\") " pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44"
Nov 04 03:26:09.434460 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.434305    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2fe2560c-8be3-41e7-a487-70c9075c72ba-cert\") pod \"kserve-controller-manager-776cc66d47-rcb44\" (UID: \"2fe2560c-8be3-41e7-a487-70c9075c72ba\") " pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44"
Nov 04 03:26:09.443254 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.443232    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Nov 04 03:26:09.443363 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.443279    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 03:26:09.443363 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.443281    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bchxs\""
Nov 04 03:26:09.444493 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.444477    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-rcb44"]
Nov 04 03:26:09.459187 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.459164    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"]
Nov 04 03:26:09.462476 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.462454    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"
Nov 04 03:26:09.511894 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.511837    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Nov 04 03:26:09.512299 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.511950    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6dtfn\""
Nov 04 03:26:09.513412 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.513387    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:09.518442 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.518422    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"]
Nov 04 03:26:09.535473 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.535447    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2fe2560c-8be3-41e7-a487-70c9075c72ba-cert\") pod \"kserve-controller-manager-776cc66d47-rcb44\" (UID: \"2fe2560c-8be3-41e7-a487-70c9075c72ba\") " pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44"
Nov 04 03:26:09.535648 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.535522    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swrzw\" (UniqueName: \"kubernetes.io/projected/42aa46d9-08ac-4f6e-aa3a-342489803c89-kube-api-access-swrzw\") pod \"llmisvc-controller-manager-869d5dcc5d-w452v\" (UID: \"42aa46d9-08ac-4f6e-aa3a-342489803c89\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"
Nov 04 03:26:09.535648 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.535602    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkj8k\" (UniqueName: \"kubernetes.io/projected/2fe2560c-8be3-41e7-a487-70c9075c72ba-kube-api-access-gkj8k\") pod \"kserve-controller-manager-776cc66d47-rcb44\" (UID: \"2fe2560c-8be3-41e7-a487-70c9075c72ba\") " pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44"
Nov 04 03:26:09.535648 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.535633    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42aa46d9-08ac-4f6e-aa3a-342489803c89-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-w452v\" (UID: \"42aa46d9-08ac-4f6e-aa3a-342489803c89\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"
Nov 04 03:26:09.537911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.537891    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2fe2560c-8be3-41e7-a487-70c9075c72ba-cert\") pod \"kserve-controller-manager-776cc66d47-rcb44\" (UID: \"2fe2560c-8be3-41e7-a487-70c9075c72ba\") " pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44"
Nov 04 03:26:09.573152 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.573129    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-tkm8m"]
Nov 04 03:26:09.576481 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.576460    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m"
Nov 04 03:26:09.594289 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.594242    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkj8k\" (UniqueName: \"kubernetes.io/projected/2fe2560c-8be3-41e7-a487-70c9075c72ba-kube-api-access-gkj8k\") pod \"kserve-controller-manager-776cc66d47-rcb44\" (UID: \"2fe2560c-8be3-41e7-a487-70c9075c72ba\") " pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44"
Nov 04 03:26:09.625552 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.625531    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Nov 04 03:26:09.625657 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.625564    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-nrp92\""
Nov 04 03:26:09.631238 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.631220    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-tkm8m"]
Nov 04 03:26:09.636119 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.636097    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42aa46d9-08ac-4f6e-aa3a-342489803c89-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-w452v\" (UID: \"42aa46d9-08ac-4f6e-aa3a-342489803c89\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"
Nov 04 03:26:09.636214 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.636135    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97v6x\" (UniqueName: \"kubernetes.io/projected/fa9c93bf-935f-4283-8304-d42e51ed5e4c-kube-api-access-97v6x\") pod \"model-serving-api-75bd89fdfc-tkm8m\" (UID: \"fa9c93bf-935f-4283-8304-d42e51ed5e4c\") " pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m"
Nov 04 03:26:09.636214 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.636159    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa9c93bf-935f-4283-8304-d42e51ed5e4c-tls-certs\") pod \"model-serving-api-75bd89fdfc-tkm8m\" (UID: \"fa9c93bf-935f-4283-8304-d42e51ed5e4c\") " pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m"
Nov 04 03:26:09.636327 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.636301    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swrzw\" (UniqueName: \"kubernetes.io/projected/42aa46d9-08ac-4f6e-aa3a-342489803c89-kube-api-access-swrzw\") pod \"llmisvc-controller-manager-869d5dcc5d-w452v\" (UID: \"42aa46d9-08ac-4f6e-aa3a-342489803c89\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"
Nov 04 03:26:09.638332 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.638312    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42aa46d9-08ac-4f6e-aa3a-342489803c89-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-w452v\" (UID: \"42aa46d9-08ac-4f6e-aa3a-342489803c89\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"
Nov 04 03:26:09.660218 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.660195    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44"
Nov 04 03:26:09.693805 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.693784    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swrzw\" (UniqueName: \"kubernetes.io/projected/42aa46d9-08ac-4f6e-aa3a-342489803c89-kube-api-access-swrzw\") pod \"llmisvc-controller-manager-869d5dcc5d-w452v\" (UID: \"42aa46d9-08ac-4f6e-aa3a-342489803c89\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"
Nov 04 03:26:09.737509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.737474    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97v6x\" (UniqueName: \"kubernetes.io/projected/fa9c93bf-935f-4283-8304-d42e51ed5e4c-kube-api-access-97v6x\") pod \"model-serving-api-75bd89fdfc-tkm8m\" (UID: \"fa9c93bf-935f-4283-8304-d42e51ed5e4c\") " pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m"
Nov 04 03:26:09.737667 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.737524    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa9c93bf-935f-4283-8304-d42e51ed5e4c-tls-certs\") pod \"model-serving-api-75bd89fdfc-tkm8m\" (UID: \"fa9c93bf-935f-4283-8304-d42e51ed5e4c\") " pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m"
Nov 04 03:26:09.739830 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.739807    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa9c93bf-935f-4283-8304-d42e51ed5e4c-tls-certs\") pod \"model-serving-api-75bd89fdfc-tkm8m\" (UID: \"fa9c93bf-935f-4283-8304-d42e51ed5e4c\") " pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m"
Nov 04 03:26:09.775001 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.774975    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"
Nov 04 03:26:09.795530 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.795496    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97v6x\" (UniqueName: \"kubernetes.io/projected/fa9c93bf-935f-4283-8304-d42e51ed5e4c-kube-api-access-97v6x\") pod \"model-serving-api-75bd89fdfc-tkm8m\" (UID: \"fa9c93bf-935f-4283-8304-d42e51ed5e4c\") " pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m"
Nov 04 03:26:09.889207 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:09.889126    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m"
Nov 04 03:26:10.025463 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:10.025433    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-rcb44"]
Nov 04 03:26:10.032875 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:26:10.032831    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fe2560c_8be3_41e7_a487_70c9075c72ba.slice/crio-68a2d58c26fe49cf1b3a1162e58832662a02eba50a166a3885c3a6175c98d17c WatchSource:0}: Error finding container 68a2d58c26fe49cf1b3a1162e58832662a02eba50a166a3885c3a6175c98d17c: Status 404 returned error can't find the container with id 68a2d58c26fe49cf1b3a1162e58832662a02eba50a166a3885c3a6175c98d17c
Nov 04 03:26:10.141687 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:10.141618    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"]
Nov 04 03:26:10.148287 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:26:10.148262    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod42aa46d9_08ac_4f6e_aa3a_342489803c89.slice/crio-042d086847cd98fad97d03579542345f4f41b450b1e29e33b0264f846ca0224e WatchSource:0}: Error finding container 042d086847cd98fad97d03579542345f4f41b450b1e29e33b0264f846ca0224e: Status 404 returned error can't find the container with id 042d086847cd98fad97d03579542345f4f41b450b1e29e33b0264f846ca0224e
Nov 04 03:26:10.255115 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:10.255083    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-tkm8m"]
Nov 04 03:26:10.262433 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:26:10.262404    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9c93bf_935f_4283_8304_d42e51ed5e4c.slice/crio-db8a639f98a91bf73e93e3f7553be05ad7bcea041ab49aa2de0bcba086114861 WatchSource:0}: Error finding container db8a639f98a91bf73e93e3f7553be05ad7bcea041ab49aa2de0bcba086114861: Status 404 returned error can't find the container with id db8a639f98a91bf73e93e3f7553be05ad7bcea041ab49aa2de0bcba086114861
Nov 04 03:26:10.318413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:10.318379    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m" event={"ID":"fa9c93bf-935f-4283-8304-d42e51ed5e4c","Type":"ContainerStarted","Data":"db8a639f98a91bf73e93e3f7553be05ad7bcea041ab49aa2de0bcba086114861"}
Nov 04 03:26:10.319559 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:10.319525    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v" event={"ID":"42aa46d9-08ac-4f6e-aa3a-342489803c89","Type":"ContainerStarted","Data":"042d086847cd98fad97d03579542345f4f41b450b1e29e33b0264f846ca0224e"}
Nov 04 03:26:10.320630 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:10.320609    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44" event={"ID":"2fe2560c-8be3-41e7-a487-70c9075c72ba","Type":"ContainerStarted","Data":"68a2d58c26fe49cf1b3a1162e58832662a02eba50a166a3885c3a6175c98d17c"}
Nov 04 03:26:10.727752 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:10.727716    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:13.337002 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:13.336973    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m" event={"ID":"fa9c93bf-935f-4283-8304-d42e51ed5e4c","Type":"ContainerStarted","Data":"e68551bd51bb5c57111e3f9da1638279020e8f59e102ebeab9b680d7aa6a2b0a"}
Nov 04 03:26:13.337347 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:13.337110    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m"
Nov 04 03:26:13.338421 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:13.338396    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v" event={"ID":"42aa46d9-08ac-4f6e-aa3a-342489803c89","Type":"ContainerStarted","Data":"dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122"}
Nov 04 03:26:13.338529 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:13.338514    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"
Nov 04 03:26:13.447411 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:13.447305    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m" podStartSLOduration=1.900125519 podStartE2EDuration="4.447287907s" totalImagesPullingTime="2.547162388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:26:13.445746567 +0000 UTC m=+543.260537851" watchObservedRunningTime="2026-11-04 03:26:13.447287907 +0000 UTC m=+543.262079184"
Nov 04 03:26:13.558211 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:13.558159    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v" podStartSLOduration=1.5433158649999998 podStartE2EDuration="4.558143953s" totalImagesPullingTime="3.014828088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:26:13.556724142 +0000 UTC m=+543.371515421" watchObservedRunningTime="2026-11-04 03:26:13.558143953 +0000 UTC m=+543.372935230"
Nov 04 03:26:14.343762 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:14.343722    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44" event={"ID":"2fe2560c-8be3-41e7-a487-70c9075c72ba","Type":"ContainerStarted","Data":"939b439e58243ba9c9c3b41a14cb75b7a65ad2151b073496e42938a0c68dc405"}
Nov 04 03:26:14.344289 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:14.344134    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44"
Nov 04 03:26:14.456176 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:14.456120    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44" podStartSLOduration=2.182119165 podStartE2EDuration="5.456107657s" totalImagesPullingTime="3.273988492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:26:14.455121993 +0000 UTC m=+544.269913272" watchObservedRunningTime="2026-11-04 03:26:14.456107657 +0000 UTC m=+544.270898934"
Nov 04 03:26:14.924361 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:14.924323    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:15.076828 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:15.076790    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:15.919122 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:15.919083    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:16.067503 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:16.067474    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:16.228507 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:16.228418    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:16.388710 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:16.388677    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:16.548899 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:16.548863    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:16.708881 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:16.708825    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:16.908952 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:16.908839    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:17.111488 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:17.111450    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:17.263442 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:17.263409    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:17.440650 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:17.440617    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:24.348534 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:24.348498    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-tkm8m"
Nov 04 03:26:25.956983 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:25.956947    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:26:31.679428 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:31.679397    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:31.978811 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:31.978738    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-9dzz8"]
Nov 04 03:26:31.982387 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:31.982369    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-9dzz8"
Nov 04 03:26:32.028582 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.028549    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:32.037084 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.037061    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-9dzz8"]
Nov 04 03:26:32.127494 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.127458    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7pw2\" (UniqueName: \"kubernetes.io/projected/c3db8be8-7bb8-4575-b9f9-50693812aae3-kube-api-access-x7pw2\") pod \"authorino-6f6944bcdd-9dzz8\" (UID: \"c3db8be8-7bb8-4575-b9f9-50693812aae3\") " pod="kuadrant-system/authorino-6f6944bcdd-9dzz8"
Nov 04 03:26:32.228472 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.228437    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7pw2\" (UniqueName: \"kubernetes.io/projected/c3db8be8-7bb8-4575-b9f9-50693812aae3-kube-api-access-x7pw2\") pod \"authorino-6f6944bcdd-9dzz8\" (UID: \"c3db8be8-7bb8-4575-b9f9-50693812aae3\") " pod="kuadrant-system/authorino-6f6944bcdd-9dzz8"
Nov 04 03:26:32.286097 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.286061    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7pw2\" (UniqueName: \"kubernetes.io/projected/c3db8be8-7bb8-4575-b9f9-50693812aae3-kube-api-access-x7pw2\") pod \"authorino-6f6944bcdd-9dzz8\" (UID: \"c3db8be8-7bb8-4575-b9f9-50693812aae3\") " pod="kuadrant-system/authorino-6f6944bcdd-9dzz8"
Nov 04 03:26:32.293255 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.293231    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-9dzz8"
Nov 04 03:26:32.336013 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.335980    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-9dzz8"]
Nov 04 03:26:32.363862 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.363719    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76f555cd65-p5s8m"]
Nov 04 03:26:32.366964 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.366943    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76f555cd65-p5s8m"
Nov 04 03:26:32.420307 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.420281    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76f555cd65-p5s8m"]
Nov 04 03:26:32.430386 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.430359    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssdpw\" (UniqueName: \"kubernetes.io/projected/0dbf1906-e86c-43ec-b09d-97ecfa9e6866-kube-api-access-ssdpw\") pod \"authorino-76f555cd65-p5s8m\" (UID: \"0dbf1906-e86c-43ec-b09d-97ecfa9e6866\") " pod="kuadrant-system/authorino-76f555cd65-p5s8m"
Nov 04 03:26:32.531275 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.531245    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssdpw\" (UniqueName: \"kubernetes.io/projected/0dbf1906-e86c-43ec-b09d-97ecfa9e6866-kube-api-access-ssdpw\") pod \"authorino-76f555cd65-p5s8m\" (UID: \"0dbf1906-e86c-43ec-b09d-97ecfa9e6866\") " pod="kuadrant-system/authorino-76f555cd65-p5s8m"
Nov 04 03:26:32.589171 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.589106    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssdpw\" (UniqueName: \"kubernetes.io/projected/0dbf1906-e86c-43ec-b09d-97ecfa9e6866-kube-api-access-ssdpw\") pod \"authorino-76f555cd65-p5s8m\" (UID: \"0dbf1906-e86c-43ec-b09d-97ecfa9e6866\") " pod="kuadrant-system/authorino-76f555cd65-p5s8m"
Nov 04 03:26:32.648872 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:26:32.648743    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:26:32.659331 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.659303    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-9dzz8"]
Nov 04 03:26:32.666376 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:26:32.666351    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3db8be8_7bb8_4575_b9f9_50693812aae3.slice/crio-6d4023fbc75b3a5f3168887b6850e4d83f50b0a6d0332680a77f0fcdb52b93a5 WatchSource:0}: Error finding container 6d4023fbc75b3a5f3168887b6850e4d83f50b0a6d0332680a77f0fcdb52b93a5: Status 404 returned error can't find the container with id 6d4023fbc75b3a5f3168887b6850e4d83f50b0a6d0332680a77f0fcdb52b93a5
Nov 04 03:26:32.677522 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.677500    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76f555cd65-p5s8m"
Nov 04 03:26:32.706763 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:32.706731    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76f555cd65-p5s8m"]
Nov 04 03:26:33.042975 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:33.042941    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76f555cd65-p5s8m"]
Nov 04 03:26:33.049594 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:26:33.049560    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0dbf1906_e86c_43ec_b09d_97ecfa9e6866.slice/crio-94ce29bcb4f929889e1f2bc869e37af1577acb86d1a0f920832472797010ef56 WatchSource:0}: Error finding container 94ce29bcb4f929889e1f2bc869e37af1577acb86d1a0f920832472797010ef56: Status 404 returned error can't find the container with id 94ce29bcb4f929889e1f2bc869e37af1577acb86d1a0f920832472797010ef56
Nov 04 03:26:33.423794 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:33.423712    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-9dzz8" event={"ID":"c3db8be8-7bb8-4575-b9f9-50693812aae3","Type":"ContainerStarted","Data":"6d4023fbc75b3a5f3168887b6850e4d83f50b0a6d0332680a77f0fcdb52b93a5"}
Nov 04 03:26:33.424808 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:33.424782    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76f555cd65-p5s8m" event={"ID":"0dbf1906-e86c-43ec-b09d-97ecfa9e6866","Type":"ContainerStarted","Data":"94ce29bcb4f929889e1f2bc869e37af1577acb86d1a0f920832472797010ef56"}
Nov 04 03:26:35.435578 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.435533    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-9dzz8" event={"ID":"c3db8be8-7bb8-4575-b9f9-50693812aae3","Type":"ContainerStarted","Data":"f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387"}
Nov 04 03:26:35.436054 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.435610    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6f6944bcdd-9dzz8" podUID="c3db8be8-7bb8-4575-b9f9-50693812aae3" containerName="authorino" containerID="cri-o://f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387" gracePeriod=30
Nov 04 03:26:35.437413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.437387    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76f555cd65-p5s8m" event={"ID":"0dbf1906-e86c-43ec-b09d-97ecfa9e6866","Type":"ContainerStarted","Data":"80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9"}
Nov 04 03:26:35.437541 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.437424    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-76f555cd65-p5s8m" podUID="0dbf1906-e86c-43ec-b09d-97ecfa9e6866" containerName="authorino" containerID="cri-o://80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9" gracePeriod=30
Nov 04 03:26:35.545426 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.545373    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6f6944bcdd-9dzz8" podStartSLOduration=2.759771343 podStartE2EDuration="4.545358092s" totalImagesPullingTime="1.785586749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:26:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:26:35.543431326 +0000 UTC m=+565.358222603" watchObservedRunningTime="2026-11-04 03:26:35.545358092 +0000 UTC m=+565.360149369"
Nov 04 03:26:35.717826 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.717801    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-9dzz8"
Nov 04 03:26:35.719973 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.719953    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76f555cd65-p5s8m"
Nov 04 03:26:35.836620 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.836569    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-76f555cd65-p5s8m" podStartSLOduration=2.3124134 podStartE2EDuration="3.83655386s" totalImagesPullingTime="1.52414046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:26:35.653035858 +0000 UTC m=+565.467827136" watchObservedRunningTime="2026-11-04 03:26:35.83655386 +0000 UTC m=+565.651345131"
Nov 04 03:26:35.864858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.864814    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0dbf1906-e86c-43ec-b09d-97ecfa9e6866-kube-api-access-ssdpw\" (UniqueName: \"kubernetes.io/projected/0dbf1906-e86c-43ec-b09d-97ecfa9e6866-kube-api-access-ssdpw\") pod \"0dbf1906-e86c-43ec-b09d-97ecfa9e6866\" (UID: \"0dbf1906-e86c-43ec-b09d-97ecfa9e6866\") "
Nov 04 03:26:35.865022 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.864933    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3db8be8-7bb8-4575-b9f9-50693812aae3-kube-api-access-x7pw2\" (UniqueName: \"kubernetes.io/projected/c3db8be8-7bb8-4575-b9f9-50693812aae3-kube-api-access-x7pw2\") pod \"c3db8be8-7bb8-4575-b9f9-50693812aae3\" (UID: \"c3db8be8-7bb8-4575-b9f9-50693812aae3\") "
Nov 04 03:26:35.867099 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.867046    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3db8be8-7bb8-4575-b9f9-50693812aae3-kube-api-access-x7pw2" pod "c3db8be8-7bb8-4575-b9f9-50693812aae3" (UID: "c3db8be8-7bb8-4575-b9f9-50693812aae3"). InnerVolumeSpecName "kube-api-access-x7pw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:26:35.867237 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.867112    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dbf1906-e86c-43ec-b09d-97ecfa9e6866-kube-api-access-ssdpw" pod "0dbf1906-e86c-43ec-b09d-97ecfa9e6866" (UID: "0dbf1906-e86c-43ec-b09d-97ecfa9e6866"). InnerVolumeSpecName "kube-api-access-ssdpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:26:35.899593 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.899562    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-2f296"]
Nov 04 03:26:35.899758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.899739    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-2f296" podUID="ae6d187a-3d9b-400a-a31a-70ccb0a0ccae" containerName="authorino" containerID="cri-o://826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7" gracePeriod=30
Nov 04 03:26:35.966501 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.966438    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssdpw\" (UniqueName: \"kubernetes.io/projected/0dbf1906-e86c-43ec-b09d-97ecfa9e6866-kube-api-access-ssdpw\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:35.966501 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:35.966463    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7pw2\" (UniqueName: \"kubernetes.io/projected/c3db8be8-7bb8-4575-b9f9-50693812aae3-kube-api-access-x7pw2\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:36.135526 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.135502    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-2f296"
Nov 04 03:26:36.268536 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.268509    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae6d187a-3d9b-400a-a31a-70ccb0a0ccae-kube-api-access-gvkml\" (UniqueName: \"kubernetes.io/projected/ae6d187a-3d9b-400a-a31a-70ccb0a0ccae-kube-api-access-gvkml\") pod \"ae6d187a-3d9b-400a-a31a-70ccb0a0ccae\" (UID: \"ae6d187a-3d9b-400a-a31a-70ccb0a0ccae\") "
Nov 04 03:26:36.270532 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.270511    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae6d187a-3d9b-400a-a31a-70ccb0a0ccae-kube-api-access-gvkml" pod "ae6d187a-3d9b-400a-a31a-70ccb0a0ccae" (UID: "ae6d187a-3d9b-400a-a31a-70ccb0a0ccae"). InnerVolumeSpecName "kube-api-access-gvkml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:26:36.369339 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.369312    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvkml\" (UniqueName: \"kubernetes.io/projected/ae6d187a-3d9b-400a-a31a-70ccb0a0ccae-kube-api-access-gvkml\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:26:36.443158 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.443121    4420 generic.go:356] "Generic (PLEG): container finished" podID="c3db8be8-7bb8-4575-b9f9-50693812aae3" containerID="f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387" exitCode=0
Nov 04 03:26:36.443593 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.443167    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-9dzz8"
Nov 04 03:26:36.443593 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.443208    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-9dzz8" event={"ID":"c3db8be8-7bb8-4575-b9f9-50693812aae3","Type":"ContainerDied","Data":"f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387"}
Nov 04 03:26:36.443593 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.443250    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-9dzz8" event={"ID":"c3db8be8-7bb8-4575-b9f9-50693812aae3","Type":"ContainerDied","Data":"6d4023fbc75b3a5f3168887b6850e4d83f50b0a6d0332680a77f0fcdb52b93a5"}
Nov 04 03:26:36.443593 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.443268    4420 scope.go:122] "RemoveContainer" containerID="f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387"
Nov 04 03:26:36.444436 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.444411    4420 generic.go:356] "Generic (PLEG): container finished" podID="0dbf1906-e86c-43ec-b09d-97ecfa9e6866" containerID="80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9" exitCode=0
Nov 04 03:26:36.444528 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.444454    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76f555cd65-p5s8m"
Nov 04 03:26:36.444528 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.444508    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76f555cd65-p5s8m" event={"ID":"0dbf1906-e86c-43ec-b09d-97ecfa9e6866","Type":"ContainerDied","Data":"80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9"}
Nov 04 03:26:36.444643 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.444546    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76f555cd65-p5s8m" event={"ID":"0dbf1906-e86c-43ec-b09d-97ecfa9e6866","Type":"ContainerDied","Data":"94ce29bcb4f929889e1f2bc869e37af1577acb86d1a0f920832472797010ef56"}
Nov 04 03:26:36.446033 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.446008    4420 generic.go:356] "Generic (PLEG): container finished" podID="ae6d187a-3d9b-400a-a31a-70ccb0a0ccae" containerID="826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7" exitCode=0
Nov 04 03:26:36.446141 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.446051    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-2f296"
Nov 04 03:26:36.446141 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.446095    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-2f296" event={"ID":"ae6d187a-3d9b-400a-a31a-70ccb0a0ccae","Type":"ContainerDied","Data":"826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7"}
Nov 04 03:26:36.446213 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.446141    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-2f296" event={"ID":"ae6d187a-3d9b-400a-a31a-70ccb0a0ccae","Type":"ContainerDied","Data":"5da6c811667cf39dd4ad6791289fdcc52b2300d5570721ddc9b288fa18549e99"}
Nov 04 03:26:36.452161 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.452144    4420 scope.go:122] "RemoveContainer" containerID="f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387"
Nov 04 03:26:36.452425 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:26:36.452406    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387\": container with ID starting with f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387 not found: ID does not exist" containerID="f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387"
Nov 04 03:26:36.452497 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.452436    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387"} err="failed to get container status \"f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387\": rpc error: code = NotFound desc = could not find container \"f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387\": container with ID starting with f6f2a4fd0a9d130886cfb5d471b7a18edf7281d0c692da036b831274602e3387 not found: ID does not exist"
Nov 04 03:26:36.452497 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.452457    4420 scope.go:122] "RemoveContainer" containerID="80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9"
Nov 04 03:26:36.459523 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.459506    4420 scope.go:122] "RemoveContainer" containerID="80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9"
Nov 04 03:26:36.459789 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:26:36.459771    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9\": container with ID starting with 80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9 not found: ID does not exist" containerID="80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9"
Nov 04 03:26:36.459833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.459796    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9"} err="failed to get container status \"80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9\": rpc error: code = NotFound desc = could not find container \"80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9\": container with ID starting with 80a7a397d1a89078d650e853253be2c134d8de03916176e7f1fbed420c57dfb9 not found: ID does not exist"
Nov 04 03:26:36.459833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.459814    4420 scope.go:122] "RemoveContainer" containerID="826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7"
Nov 04 03:26:36.466758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.466740    4420 scope.go:122] "RemoveContainer" containerID="826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7"
Nov 04 03:26:36.467029 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:26:36.467012    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7\": container with ID starting with 826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7 not found: ID does not exist" containerID="826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7"
Nov 04 03:26:36.467080 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.467037    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7"} err="failed to get container status \"826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7\": rpc error: code = NotFound desc = could not find container \"826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7\": container with ID starting with 826e8120c1445bc9612cf9193334ba3f50d83fe8063de2d41aba71f43addabd7 not found: ID does not exist"
Nov 04 03:26:36.596564 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.596488    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-9dzz8"]
Nov 04 03:26:36.600301 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.600277    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-9dzz8"]
Nov 04 03:26:36.704409 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.704379    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76f555cd65-p5s8m"]
Nov 04 03:26:36.721284 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.721253    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76f555cd65-p5s8m"]
Nov 04 03:26:36.833790 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.833755    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-2f296"]
Nov 04 03:26:36.841317 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:36.841293    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-2f296"]
Nov 04 03:26:38.305001 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:38.304974    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0dbf1906-e86c-43ec-b09d-97ecfa9e6866" path="/var/lib/kubelet/pods/0dbf1906-e86c-43ec-b09d-97ecfa9e6866/volumes"
Nov 04 03:26:38.305365 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:38.305276    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae6d187a-3d9b-400a-a31a-70ccb0a0ccae" path="/var/lib/kubelet/pods/ae6d187a-3d9b-400a-a31a-70ccb0a0ccae/volumes"
Nov 04 03:26:38.305553 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:38.305541    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3db8be8-7bb8-4575-b9f9-50693812aae3" path="/var/lib/kubelet/pods/c3db8be8-7bb8-4575-b9f9-50693812aae3/volumes"
Nov 04 03:26:44.346244 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:44.346213    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"
Nov 04 03:26:46.359634 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:46.359607    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-rcb44"
Nov 04 03:26:52.742444 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:52.742359    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:52.894099 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:52.894028    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:54.517969 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:54.517937    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:54.714459 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:54.714425    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:58.492295 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:58.492253    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:26:58.690301 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:26:58.690270    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:02.649520 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:27:02.649482    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:27:03.048937 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:03.048897    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:10.330665 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:10.330630    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:27:10.333135 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:10.333114    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:27:17.179747 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.179711    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"]
Nov 04 03:27:17.180162 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.180056    4420 cpu_manager.go:414] "containerMap: removing container" podUID="0dbf1906-e86c-43ec-b09d-97ecfa9e6866" containerName="authorino"
Nov 04 03:27:17.180162 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.180068    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0dbf1906-e86c-43ec-b09d-97ecfa9e6866" containerName="authorino"
Nov 04 03:27:17.180162 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.180080    4420 cpu_manager.go:414] "containerMap: removing container" podUID="ae6d187a-3d9b-400a-a31a-70ccb0a0ccae" containerName="authorino"
Nov 04 03:27:17.180162 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.180085    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae6d187a-3d9b-400a-a31a-70ccb0a0ccae" containerName="authorino"
Nov 04 03:27:17.180162 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.180129    4420 cpu_manager.go:414] "containerMap: removing container" podUID="c3db8be8-7bb8-4575-b9f9-50693812aae3" containerName="authorino"
Nov 04 03:27:17.180162 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.180135    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3db8be8-7bb8-4575-b9f9-50693812aae3" containerName="authorino"
Nov 04 03:27:17.180345 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.180189    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3db8be8-7bb8-4575-b9f9-50693812aae3" containerName="authorino"
Nov 04 03:27:17.180345 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.180196    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="0dbf1906-e86c-43ec-b09d-97ecfa9e6866" containerName="authorino"
Nov 04 03:27:17.180345 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.180205    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae6d187a-3d9b-400a-a31a-70ccb0a0ccae" containerName="authorino"
Nov 04 03:27:17.184146 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.184126    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"
Nov 04 03:27:17.244108 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.244078    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"]
Nov 04 03:27:17.285472 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.285424    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d26d201-c93f-479b-8975-dacdbfe056e2-cert\") pod \"llmisvc-controller-manager-6466c554d8-fbbwk\" (UID: \"7d26d201-c93f-479b-8975-dacdbfe056e2\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"
Nov 04 03:27:17.285629 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.285549    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtn5k\" (UniqueName: \"kubernetes.io/projected/7d26d201-c93f-479b-8975-dacdbfe056e2-kube-api-access-jtn5k\") pod \"llmisvc-controller-manager-6466c554d8-fbbwk\" (UID: \"7d26d201-c93f-479b-8975-dacdbfe056e2\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"
Nov 04 03:27:17.386224 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.386194    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtn5k\" (UniqueName: \"kubernetes.io/projected/7d26d201-c93f-479b-8975-dacdbfe056e2-kube-api-access-jtn5k\") pod \"llmisvc-controller-manager-6466c554d8-fbbwk\" (UID: \"7d26d201-c93f-479b-8975-dacdbfe056e2\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"
Nov 04 03:27:17.386371 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.386247    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d26d201-c93f-479b-8975-dacdbfe056e2-cert\") pod \"llmisvc-controller-manager-6466c554d8-fbbwk\" (UID: \"7d26d201-c93f-479b-8975-dacdbfe056e2\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"
Nov 04 03:27:17.388551 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.388531    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d26d201-c93f-479b-8975-dacdbfe056e2-cert\") pod \"llmisvc-controller-manager-6466c554d8-fbbwk\" (UID: \"7d26d201-c93f-479b-8975-dacdbfe056e2\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"
Nov 04 03:27:17.444285 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.444217    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtn5k\" (UniqueName: \"kubernetes.io/projected/7d26d201-c93f-479b-8975-dacdbfe056e2-kube-api-access-jtn5k\") pod \"llmisvc-controller-manager-6466c554d8-fbbwk\" (UID: \"7d26d201-c93f-479b-8975-dacdbfe056e2\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"
Nov 04 03:27:17.501690 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.501662    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"
Nov 04 03:27:17.886973 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:17.886940    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"]
Nov 04 03:27:17.893503 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:27:17.893474    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d26d201_c93f_479b_8975_dacdbfe056e2.slice/crio-5510e3534193bf211f66652cde9bb0946af2403771976e1c6917278bd855255d WatchSource:0}: Error finding container 5510e3534193bf211f66652cde9bb0946af2403771976e1c6917278bd855255d: Status 404 returned error can't find the container with id 5510e3534193bf211f66652cde9bb0946af2403771976e1c6917278bd855255d
Nov 04 03:27:18.626617 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:18.626583    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk" event={"ID":"7d26d201-c93f-479b-8975-dacdbfe056e2","Type":"ContainerStarted","Data":"5510e3534193bf211f66652cde9bb0946af2403771976e1c6917278bd855255d"}
Nov 04 03:27:19.632989 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:19.632945    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk" event={"ID":"7d26d201-c93f-479b-8975-dacdbfe056e2","Type":"ContainerStarted","Data":"5d45e397c628cafeb449a53947a1f29f4bc3a62a53771cad34f4a3815d8928af"}
Nov 04 03:27:19.633441 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:19.633087    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"
Nov 04 03:27:19.742374 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:19.742325    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk" podStartSLOduration=1.844735369 podStartE2EDuration="2.742311463s" totalImagesPullingTime="897.576094ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:27:19.741954685 +0000 UTC m=+609.556745974" watchObservedRunningTime="2026-11-04 03:27:19.742311463 +0000 UTC m=+609.557102740"
Nov 04 03:27:32.648726 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:27:32.648691    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:27:50.639915 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:50.639887    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-fbbwk"
Nov 04 03:27:50.798162 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:50.798131    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"]
Nov 04 03:27:50.798443 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:50.798416    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v" podUID="42aa46d9-08ac-4f6e-aa3a-342489803c89" containerName="manager" containerID="cri-o://dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122" gracePeriod=30
Nov 04 03:27:51.044310 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.044286    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"
Nov 04 03:27:51.159758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.159726    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42aa46d9-08ac-4f6e-aa3a-342489803c89-cert\" (UniqueName: \"kubernetes.io/secret/42aa46d9-08ac-4f6e-aa3a-342489803c89-cert\") pod \"42aa46d9-08ac-4f6e-aa3a-342489803c89\" (UID: \"42aa46d9-08ac-4f6e-aa3a-342489803c89\") "
Nov 04 03:27:51.159966 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.159838    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42aa46d9-08ac-4f6e-aa3a-342489803c89-kube-api-access-swrzw\" (UniqueName: \"kubernetes.io/projected/42aa46d9-08ac-4f6e-aa3a-342489803c89-kube-api-access-swrzw\") pod \"42aa46d9-08ac-4f6e-aa3a-342489803c89\" (UID: \"42aa46d9-08ac-4f6e-aa3a-342489803c89\") "
Nov 04 03:27:51.162338 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.162311    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42aa46d9-08ac-4f6e-aa3a-342489803c89-cert" pod "42aa46d9-08ac-4f6e-aa3a-342489803c89" (UID: "42aa46d9-08ac-4f6e-aa3a-342489803c89"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:27:51.162509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.162482    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42aa46d9-08ac-4f6e-aa3a-342489803c89-kube-api-access-swrzw" pod "42aa46d9-08ac-4f6e-aa3a-342489803c89" (UID: "42aa46d9-08ac-4f6e-aa3a-342489803c89"). InnerVolumeSpecName "kube-api-access-swrzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:27:51.260451 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.260421    4420 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42aa46d9-08ac-4f6e-aa3a-342489803c89-cert\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:27:51.260451 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.260449    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swrzw\" (UniqueName: \"kubernetes.io/projected/42aa46d9-08ac-4f6e-aa3a-342489803c89-kube-api-access-swrzw\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:27:51.772645 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.772610    4420 generic.go:356] "Generic (PLEG): container finished" podID="42aa46d9-08ac-4f6e-aa3a-342489803c89" containerID="dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122" exitCode=0
Nov 04 03:27:51.773094 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.772672    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"
Nov 04 03:27:51.773094 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.772697    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v" event={"ID":"42aa46d9-08ac-4f6e-aa3a-342489803c89","Type":"ContainerDied","Data":"dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122"}
Nov 04 03:27:51.773094 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.772750    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v" event={"ID":"42aa46d9-08ac-4f6e-aa3a-342489803c89","Type":"ContainerDied","Data":"042d086847cd98fad97d03579542345f4f41b450b1e29e33b0264f846ca0224e"}
Nov 04 03:27:51.773094 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.772773    4420 scope.go:122] "RemoveContainer" containerID="dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122"
Nov 04 03:27:51.781323 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.781303    4420 scope.go:122] "RemoveContainer" containerID="dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122"
Nov 04 03:27:51.781587 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:27:51.781570    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122\": container with ID starting with dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122 not found: ID does not exist" containerID="dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122"
Nov 04 03:27:51.781632 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.781596    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122"} err="failed to get container status \"dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122\": rpc error: code = NotFound desc = could not find container \"dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122\": container with ID starting with dd42f0594b884764dbc66a36ef07b3f64067104d6c3f13afed9930ed7221f122 not found: ID does not exist"
Nov 04 03:27:51.932962 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.932926    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"]
Nov 04 03:27:51.937094 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:51.937066    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-w452v"]
Nov 04 03:27:52.015942 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:52.015905    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:52.194171 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:52.194086    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:52.304796 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:52.304769    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42aa46d9-08ac-4f6e-aa3a-342489803c89" path="/var/lib/kubelet/pods/42aa46d9-08ac-4f6e-aa3a-342489803c89/volumes"
Nov 04 03:27:59.182190 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:59.182150    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:27:59.377693 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:27:59.377661    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:02.648109 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:28:02.648066    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:28:03.401746 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:03.401707    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:21.377437 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:21.377363    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:21.799422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:21.799393    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:23.057945 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:23.057904    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:23.259636 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:23.259597    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:32.649363 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:28:32.649322    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:28:34.031988 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:34.031947    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:36.770810 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:36.770774    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:28:41.423435 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:41.423400    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:45.229179 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:45.229141    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:49.784437 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:49.784405    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:49.874067 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:49.874037    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:28:53.299059 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:53.299028    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:28:56.914449 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:28:56.914407    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:29:02.212094 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.212048    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:29:02.371553 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.371522    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"]
Nov 04 03:29:02.371888 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.371875    4420 cpu_manager.go:414] "containerMap: removing container" podUID="42aa46d9-08ac-4f6e-aa3a-342489803c89" containerName="manager"
Nov 04 03:29:02.371946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.371889    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42aa46d9-08ac-4f6e-aa3a-342489803c89" containerName="manager"
Nov 04 03:29:02.371946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.371940    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="42aa46d9-08ac-4f6e-aa3a-342489803c89" containerName="manager"
Nov 04 03:29:02.374897 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.374877    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.459992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.459961    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"]
Nov 04 03:29:02.516573 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.516541    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 03:29:02.516573 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.516541    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dk4hf\""
Nov 04 03:29:02.535974 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.535940    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.536125 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.535979    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.536125 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.536000    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48072923-8c97-4319-878d-fca3595c4d5c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.536125 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.536084    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wdcx\" (UniqueName: \"kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-kube-api-access-9wdcx\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.536299 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.536153    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.536299 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.536275    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.536404 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.536304    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48072923-8c97-4319-878d-fca3595c4d5c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.536404 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.536344    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.536404 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.536373    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.637239 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.637206    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.637413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.637245    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48072923-8c97-4319-878d-fca3595c4d5c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.637413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.637272    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.637413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.637297    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.637413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.637365    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.637413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.637400    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.637675 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.637425    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48072923-8c97-4319-878d-fca3595c4d5c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.637675 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.637448    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wdcx\" (UniqueName: \"kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-kube-api-access-9wdcx\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.637775 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.637714    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.637881 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.637857    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.637992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.637921    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.638094 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.638077    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.638189 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.638161    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.638322 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.638301    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48072923-8c97-4319-878d-fca3595c4d5c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.639686 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.639659    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.640069 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.640045    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48072923-8c97-4319-878d-fca3595c4d5c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.652533 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:29:02.652499    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:29:02.695704 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.695670    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.695868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.695790    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wdcx\" (UniqueName: \"kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-kube-api-access-9wdcx\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-rjj92\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:02.987306 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:02.987231    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:03.466059 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:03.465979    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"]
Nov 04 03:29:03.472935 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:29:03.472903    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48072923_8c97_4319_878d_fca3595c4d5c.slice/crio-69bb3ff17b49356409af9263003c24d1c2563bb84487039772c323cdda99e3fd WatchSource:0}: Error finding container 69bb3ff17b49356409af9263003c24d1c2563bb84487039772c323cdda99e3fd: Status 404 returned error can't find the container with id 69bb3ff17b49356409af9263003c24d1c2563bb84487039772c323cdda99e3fd
Nov 04 03:29:03.474885 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:03.474869    4420 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:29:03.741244 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:03.741208    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:29:04.064170 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:04.064090    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92" event={"ID":"48072923-8c97-4319-878d-fca3595c4d5c","Type":"ContainerStarted","Data":"69bb3ff17b49356409af9263003c24d1c2563bb84487039772c323cdda99e3fd"}
Nov 04 03:29:05.114282 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:05.114249    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:29:07.380198 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:07.380167    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:29:07.380459 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:07.380232    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:29:07.380459 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:07.380262    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:29:08.084762 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:08.084730    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92" event={"ID":"48072923-8c97-4319-878d-fca3595c4d5c","Type":"ContainerStarted","Data":"70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d"}
Nov 04 03:29:08.258867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:08.249663    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92" podStartSLOduration=2.345331276 podStartE2EDuration="6.249646249s" totalImagesPullingTime="3.904314973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:29:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:29:08.249286502 +0000 UTC m=+718.064077780" watchObservedRunningTime="2026-11-04 03:29:08.249646249 +0000 UTC m=+718.064437507"
Nov 04 03:29:08.418064 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:08.417977    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:29:08.988236 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:08.988200    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:08.989693 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:08.989670    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-rjj92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:29:08.989753 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:08.989719    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92" podUID="48072923-8c97-4319-878d-fca3595c4d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 04 03:29:09.987723 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:09.987687    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-rjj92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:29:09.988139 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:09.987737    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92" podUID="48072923-8c97-4319-878d-fca3595c4d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 04 03:29:10.988327 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:10.988289    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-rjj92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:29:10.988799 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:10.988347    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92" podUID="48072923-8c97-4319-878d-fca3595c4d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 04 03:29:11.987889 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:11.987829    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-rjj92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:29:11.988069 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:11.987907    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92" podUID="48072923-8c97-4319-878d-fca3595c4d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 04 03:29:12.988329 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:12.988298    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-rjj92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:29:12.988706 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:12.988361    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92" podUID="48072923-8c97-4319-878d-fca3595c4d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 04 03:29:13.987544 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:13.987509    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-rjj92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:29:13.987740 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:13.987566    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92" podUID="48072923-8c97-4319-878d-fca3595c4d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 04 03:29:15.603095 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:15.603063    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:15.603536 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:15.603471    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:15.604269 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:15.604250    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:29:18.380427 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:18.380398    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:29:32.626869 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.626824    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"]
Nov 04 03:29:32.630477 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.630456    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.650894 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:29:32.650865    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:29:32.682392 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.682370    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.682524 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.682413    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.682524 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.682441    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.682598 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.682560    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.682668 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.682648    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.682706 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.682692    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.682739 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.682709    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4lfb\" (UniqueName: \"kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-kube-api-access-j4lfb\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.682778 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.682766    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.682816 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.682799    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.755292 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.755265    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"]
Nov 04 03:29:32.783951 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.783923    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784100 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.783961    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784100 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.783994    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784100 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.784025    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784100 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.784040    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4lfb\" (UniqueName: \"kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-kube-api-access-j4lfb\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784100 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.784058    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784100 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.784083    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784387 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.784113    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784387 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.784151    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784498 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.784469    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784659 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.784631    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784728 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.784669    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784922 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.784902    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.784984 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.784936    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.786582 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.786561    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.786746 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.786729    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.842103 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.842071    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.842378 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.842334    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4lfb\" (UniqueName: \"kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-kube-api-access-j4lfb\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-smv2q\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:32.944884 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:32.944798    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:33.369515 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:33.369480    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"]
Nov 04 03:29:33.376240 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:29:33.376207    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9867a80_2334_4de4_b9d0_3ac27eeaa4ce.slice/crio-13bee1124f900ffc7d00455a2b6f2af9eb6099019a8419b0e1575e1164ea51c2 WatchSource:0}: Error finding container 13bee1124f900ffc7d00455a2b6f2af9eb6099019a8419b0e1575e1164ea51c2: Status 404 returned error can't find the container with id 13bee1124f900ffc7d00455a2b6f2af9eb6099019a8419b0e1575e1164ea51c2
Nov 04 03:29:33.378207 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:33.378172    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:29:33.378265 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:33.378245    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:29:33.378306 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:33.378272    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:29:34.196694 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:34.196661    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" event={"ID":"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce","Type":"ContainerStarted","Data":"000d1e1eeb20e91bb047634fa8c3aa0a63adbd84f8cb6842ea293fde0ff173ba"}
Nov 04 03:29:34.197112 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:34.196708    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" event={"ID":"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce","Type":"ContainerStarted","Data":"13bee1124f900ffc7d00455a2b6f2af9eb6099019a8419b0e1575e1164ea51c2"}
Nov 04 03:29:34.416014 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:34.415963    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" podStartSLOduration=2.415948133 podStartE2EDuration="2.415948133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:29:34.368267623 +0000 UTC m=+744.183058901" watchObservedRunningTime="2026-11-04 03:29:34.415948133 +0000 UTC m=+744.230739487"
Nov 04 03:29:34.952868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:34.948140    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:34.969996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:34.969966    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-smv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:29:34.970165 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:34.970037    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" podUID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 04 03:29:35.959237 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:35.959200    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-smv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:29:35.959806 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:35.959774    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" podUID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 04 03:29:36.964866 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:36.961207    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-smv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:29:36.964866 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:36.961280    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" podUID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 04 03:29:37.307086 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:37.307049    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:29:37.464867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:37.464224    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:29:37.962869 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:37.957229    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-smv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:29:37.962869 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:37.957292    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" podUID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 04 03:29:38.948874 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:38.948217    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-smv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:29:38.948874 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:38.948280    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" podUID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 04 03:29:39.945879 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:39.945822    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-smv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:29:39.946128 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:39.945905    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" podUID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 04 03:29:40.946120 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:40.946079    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-smv2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:29:40.946517 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:40.946137    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" podUID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 04 03:29:41.948681 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:41.948648    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:41.949123 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:41.948934    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:41.949624 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:41.949601    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:29:48.839286 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:48.839243    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:29:48.988136 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:29:48.988099    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:02.649273 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:30:02.649200    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:30:04.091550 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:04.091514    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:32.650073 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:30:32.650030    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:30:47.319028 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:47.318992    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:47.522254 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:47.522210    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:47.960783 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:47.960745    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:48.166243 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:48.166204    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:48.317192 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:48.317154    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:48.473579 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:48.473540    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:52.284905 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:52.284866    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:52.579813 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:52.579726    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:52.737906 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:52.737872    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:53.021248 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:53.021217    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:53.188151 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:53.188107    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:53.387893 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:53.387767    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:54.445712 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:54.445667    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:54.693121 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:54.693085    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:54.845910 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:54.845861    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:55.147458 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:55.147375    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:55.581392 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:55.581353    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:55.755798 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:55.755752    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:55.903544 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:55.903469    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:56.216615 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:56.216529    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:56.506501 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:56.506468    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:57.934902 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:57.934863    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:58.138946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:58.138913    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:58.758121 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:58.758089    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:58.953183 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:58.953143    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:30:59.468087 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:30:59.468057    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:00.082509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:00.082474    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:00.366923 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:00.366820    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:00.523167 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:00.523131    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:00.671794 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:00.671713    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:00.832012 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:00.831976    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:02.610891 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:02.610822    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:02.650211 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:31:02.650177    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:31:03.348259 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:03.348223    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:03.545269 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:03.545235    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:03.696569 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:03.696490    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:04.109119 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:04.109077    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:04.309802 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:04.309771    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:04.456169 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:04.456087    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:06.041387 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:06.041345    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:07.609109 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:07.609080    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:07.806145 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:07.806107    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:09.240816 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:09.240777    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:09.794929 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:09.794895    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:31:10.151999 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:10.151913    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:10.352155 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:10.352118    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:14.611136 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:14.611094    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:14.775594 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:14.775563    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:14.922271 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:14.922195    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:15.088050 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:15.088017    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:15.236699 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:15.236616    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:17.315580 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:17.315543    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:20.143827 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:20.143786    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:23.895897 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:23.895799    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:31:24.336595 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:24.336552    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:24.514761 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:24.514727    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:25.880860 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:25.880811    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:26.061776 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:26.061746    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:26.218045 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:26.217953    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:27.918026 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:27.917990    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:28.119024 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:28.118991    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:31.155101 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.155069    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"]
Nov 04 03:31:31.158299 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.158280    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.193982 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.193947    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tmp-dir\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.194143 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.194007    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-home\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.194143 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.194079    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-model-cache\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.194143 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.194118    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhh6x\" (UniqueName: \"kubernetes.io/projected/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-kube-api-access-fhh6x\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.194143 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.194143    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-dshm\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.194351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.194280    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tls-certs\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.207287 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.207248    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-82c724\"/\"kube-root-ca.crt\""
Nov 04 03:31:31.207422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.207308    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-82c724\"/\"openshift-service-ca.crt\""
Nov 04 03:31:31.214570 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.214535    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:31.253868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.253816    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-82c724\"/\"simulator-w3-82c724-kserve-self-signed-certs\""
Nov 04 03:31:31.254012 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.253922    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-82c724\"/\"default-dockercfg-rsnz8\""
Nov 04 03:31:31.255338 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.255314    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"]
Nov 04 03:31:31.295325 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.295290    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tmp-dir\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.295501 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.295343    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-home\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.295501 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.295371    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-model-cache\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.295501 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.295397    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhh6x\" (UniqueName: \"kubernetes.io/projected/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-kube-api-access-fhh6x\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.295501 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.295423    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-dshm\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.295730 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.295512    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tls-certs\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.295807 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.295778    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tmp-dir\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.296161 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.296139    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-home\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.296244 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.296144    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-model-cache\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.297809 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.297787    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-dshm\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.298015 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.297997    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tls-certs\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.353081 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.353052    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhh6x\" (UniqueName: \"kubernetes.io/projected/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-kube-api-access-fhh6x\") pod \"simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") " pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.469100 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.469006    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:31.480244 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.480174    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:31.843333 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:31.843304    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"]
Nov 04 03:31:31.850744 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:31:31.850715    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85dd0445_88e0_4c88_8d9a_14fbf6bc2656.slice/crio-01c48d1576a0ce815d4559b517d989b537dc42f541b6945f844921b2c48a6b9c WatchSource:0}: Error finding container 01c48d1576a0ce815d4559b517d989b537dc42f541b6945f844921b2c48a6b9c: Status 404 returned error can't find the container with id 01c48d1576a0ce815d4559b517d989b537dc42f541b6945f844921b2c48a6b9c
Nov 04 03:31:32.587583 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:32.587548    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:31:32.652493 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:31:32.652395    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:31:32.693533 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:32.693501    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb" event={"ID":"85dd0445-88e0-4c88-8d9a-14fbf6bc2656","Type":"ContainerStarted","Data":"01c48d1576a0ce815d4559b517d989b537dc42f541b6945f844921b2c48a6b9c"}
Nov 04 03:31:34.704716 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:34.704678    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb" event={"ID":"85dd0445-88e0-4c88-8d9a-14fbf6bc2656","Type":"ContainerStarted","Data":"8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d"}
Nov 04 03:31:34.815732 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:34.815673    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb" podStartSLOduration=1.202251722 podStartE2EDuration="3.815658704s" totalImagesPullingTime="2.613406982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:34.814719446 +0000 UTC m=+864.629510723" watchObservedRunningTime="2026-11-04 03:31:34.815658704 +0000 UTC m=+864.630449981"
Nov 04 03:31:35.469346 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:35.469314    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:36.247192 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:36.247156    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:38.092049 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:38.092013    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:38.249929 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:38.249894    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:40.799135 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:40.799098    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:31:41.457090 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:41.457057    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:41.469792 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:41.469761    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:41.482722 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:41.482700    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:41.694192 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:41.694159    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:41.745754 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:41.745731    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:31:46.525714 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:46.525674    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:51.560076 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:51.560040    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:56.695709 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:56.695675    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:56.901496 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:56.901454    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:57.847099 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.847066    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"]
Nov 04 03:31:57.850646 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.850620    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:57.927860 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.927813    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-dshm\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:57.927860 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.927854    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tmp-dir\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:57.928028 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.928007    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljmf2\" (UniqueName: \"kubernetes.io/projected/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-kube-api-access-ljmf2\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:57.928066 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.928039    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-model-cache\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:57.928103 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.928072    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-home\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:57.928156 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.928142    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tls-certs\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:57.944328 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.944295    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:31:57.946373 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.946347    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-fba151\"/\"default-dockercfg-s4t97\""
Nov 04 03:31:57.946477 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.946374    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-fba151\"/\"simulator-w2-fba151-kserve-self-signed-certs\""
Nov 04 03:31:57.946477 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.946400    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-fba151\"/\"openshift-service-ca.crt\""
Nov 04 03:31:57.946477 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.946401    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-fba151\"/\"kube-root-ca.crt\""
Nov 04 03:31:57.948118 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:57.948099    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"]
Nov 04 03:31:58.028790 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.028759    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljmf2\" (UniqueName: \"kubernetes.io/projected/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-kube-api-access-ljmf2\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.028790 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.028792    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-model-cache\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.029061 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.028811    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-home\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.029061 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.028832    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tls-certs\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.029061 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.028875    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-dshm\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.029061 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.028890    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tmp-dir\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.029276 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.029255    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-home\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.029332 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.029313    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tmp-dir\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.029436 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.029414    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-model-cache\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.031201 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.031169    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-dshm\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.031445 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.031430    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tls-certs\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.086487 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.086451    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljmf2\" (UniqueName: \"kubernetes.io/projected/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-kube-api-access-ljmf2\") pod \"simulator-w2-fba151-kserve-7445bbc99c-dg7bf\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") " pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.161399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.161306    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:31:58.542721 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.542688    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"]
Nov 04 03:31:58.549373 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:31:58.549345    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ce57bc5_3ee7_48f5_8cf3_6b29379219fc.slice/crio-7901701b5e904b122f467017da6331ca1aba23474bd8076de7908a0ebe5ffd15 WatchSource:0}: Error finding container 7901701b5e904b122f467017da6331ca1aba23474bd8076de7908a0ebe5ffd15: Status 404 returned error can't find the container with id 7901701b5e904b122f467017da6331ca1aba23474bd8076de7908a0ebe5ffd15
Nov 04 03:31:58.817085 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.817002    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf" event={"ID":"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc","Type":"ContainerStarted","Data":"06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538"}
Nov 04 03:31:58.817085 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.817042    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf" event={"ID":"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc","Type":"ContainerStarted","Data":"7901701b5e904b122f467017da6331ca1aba23474bd8076de7908a0ebe5ffd15"}
Nov 04 03:31:58.896674 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.896639    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:31:58.927195 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:58.927137    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf" podStartSLOduration=1.9271225539999999 podStartE2EDuration="1.927122554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:31:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:31:58.926172967 +0000 UTC m=+888.740964244" watchObservedRunningTime="2026-11-04 03:31:58.927122554 +0000 UTC m=+888.741913830"
Nov 04 03:31:59.162157 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:31:59.162068    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:32:02.036636 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:02.036608    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:02.648757 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:32:02.648714    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:32:03.327789 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.327746    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:03.477494 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.477459    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:03.630390 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.630297    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:03.888613 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.888532    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"]
Nov 04 03:32:03.892696 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.892671    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:03.942026 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.941993    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-82c724-openshift-default-dockercfg-42cx7\""
Nov 04 03:32:03.982995 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.982960    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-token\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:03.982995 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.982996    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-credential-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:03.983193 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.983026    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjsjb\" (UniqueName: \"kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-kube-api-access-cjsjb\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:03.983193 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.983120    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-data\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:03.983193 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.983153    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-certs\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:03.983193 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.983171    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-podinfo\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:03.983319 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.983297    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-envoy\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:03.983402 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.983388    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:03.983436 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.983420    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78034d31-2927-4a4d-aefe-85df17ec82c9-istiod-ca-cert\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:03.991377 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:03.991345    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"]
Nov 04 03:32:04.084716 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.084676    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.084716 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.084719    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78034d31-2927-4a4d-aefe-85df17ec82c9-istiod-ca-cert\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.084983 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.084738    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-token\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.084983 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.084789    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-credential-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.084983 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.084833    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjsjb\" (UniqueName: \"kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-kube-api-access-cjsjb\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.085186 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.084984    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-data\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.085186 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.085029    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-certs\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.085186 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.085049    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-podinfo\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.085186 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.085102    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-envoy\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.085399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.085357    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.085454 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.085416    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-credential-socket\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.085496 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.085462    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-data\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.085496 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.085487    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-certs\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.085665 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.085634    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78034d31-2927-4a4d-aefe-85df17ec82c9-istiod-ca-cert\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.087355 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.087335    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-envoy\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.087466 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.087448    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-podinfo\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.142950 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.142881    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-token\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.143112 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.143025    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjsjb\" (UniqueName: \"kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-kube-api-access-cjsjb\") pod \"e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") " pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.205887 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.205853    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:04.628952 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.628916    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"]
Nov 04 03:32:04.635868 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:32:04.635821    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78034d31_2927_4a4d_aefe_85df17ec82c9.slice/crio-a223de7f077704ce5c7580fbe812618d0ce50945efa3adf4e77bb708831cf8f4 WatchSource:0}: Error finding container a223de7f077704ce5c7580fbe812618d0ce50945efa3adf4e77bb708831cf8f4: Status 404 returned error can't find the container with id a223de7f077704ce5c7580fbe812618d0ce50945efa3adf4e77bb708831cf8f4
Nov 04 03:32:04.638036 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.637998    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:32:04.638143 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.638083    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:32:04.638213 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.638141    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:32:04.755160 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.755120    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:04.844324 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.844286    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" event={"ID":"78034d31-2927-4a4d-aefe-85df17ec82c9","Type":"ContainerStarted","Data":"820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a"}
Nov 04 03:32:04.844540 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.844350    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" event={"ID":"78034d31-2927-4a4d-aefe-85df17ec82c9","Type":"ContainerStarted","Data":"a223de7f077704ce5c7580fbe812618d0ce50945efa3adf4e77bb708831cf8f4"}
Nov 04 03:32:04.960437 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:04.960343    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" podStartSLOduration=1.960328566 podStartE2EDuration="1.960328566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:32:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:32:04.958233325 +0000 UTC m=+894.773024601" watchObservedRunningTime="2026-11-04 03:32:04.960328566 +0000 UTC m=+894.775119837"
Nov 04 03:32:05.206784 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:05.206748    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:05.208365 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:05.208341    4420 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Nov 04 03:32:05.208471 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:05.208421    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" podUID="78034d31-2927-4a4d-aefe-85df17ec82c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Nov 04 03:32:06.206551 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:06.206515    4420 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Nov 04 03:32:06.207302 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:06.206566    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" podUID="78034d31-2927-4a4d-aefe-85df17ec82c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Nov 04 03:32:07.206264 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:07.206226    4420 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Nov 04 03:32:07.206450 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:07.206280    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" podUID="78034d31-2927-4a4d-aefe-85df17ec82c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Nov 04 03:32:07.300493 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:07.300456    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:08.163814 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:08.163093    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:32:08.179651 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:08.179619    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:32:08.206251 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:08.206220    4420 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Nov 04 03:32:08.206403 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:08.206283    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" podUID="78034d31-2927-4a4d-aefe-85df17ec82c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Nov 04 03:32:08.903359 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:08.903329    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:32:09.207096 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:09.207017    4420 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Nov 04 03:32:09.207096 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:09.207079    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" podUID="78034d31-2927-4a4d-aefe-85df17ec82c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Nov 04 03:32:10.206969 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:10.206926    4420 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Nov 04 03:32:10.207378 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:10.207009    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" podUID="78034d31-2927-4a4d-aefe-85df17ec82c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Nov 04 03:32:10.385119 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:10.385093    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:32:10.385827 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:10.385805    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:32:11.155132 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:11.155096    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:11.210500 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:11.210472    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:11.211126 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:11.211105    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:11.211772 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:11.211750    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:32:12.312873 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:12.312819    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:12.581197 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:12.581111    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:32:17.125887 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:17.125833    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:20.570244 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:20.570213    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:20.724088 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:20.724059    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:22.321090 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:22.321055    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:26.006706 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:26.006673    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:32:26.934170 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:26.934136    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"]
Nov 04 03:32:26.937234 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:26.937213    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:26.985150 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:26.985115    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-dshm\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:26.985313 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:26.985164    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-model-cache\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:26.985313 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:26.985272    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87t4v\" (UniqueName: \"kubernetes.io/projected/b747b84b-0b77-4776-9c2e-e101a00f4c21-kube-api-access-87t4v\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:26.985313 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:26.985302    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-home\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:26.985428 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:26.985386    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-tmp-dir\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:26.985428 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:26.985409    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b747b84b-0b77-4776-9c2e-e101a00f4c21-tls-certs\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:26.986344 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:26.986324    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-ee1947\"/\"openshift-service-ca.crt\""
Nov 04 03:32:27.031382 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.031347    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:27.032960 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.032936    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ee1947\"/\"default-dockercfg-d8dk9\""
Nov 04 03:32:27.033080 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.032985    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-ee1947\"/\"kube-root-ca.crt\""
Nov 04 03:32:27.033080 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.033032    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ee1947\"/\"simulator-w1-ee1947-kserve-self-signed-certs\""
Nov 04 03:32:27.034580 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.034557    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"]
Nov 04 03:32:27.086186 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.086150    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-home\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.086355 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.086206    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-tmp-dir\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.086355 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.086231    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b747b84b-0b77-4776-9c2e-e101a00f4c21-tls-certs\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.086355 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.086264    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-dshm\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.086355 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.086313    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-model-cache\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.086539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.086370    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87t4v\" (UniqueName: \"kubernetes.io/projected/b747b84b-0b77-4776-9c2e-e101a00f4c21-kube-api-access-87t4v\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.086625 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.086599    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-home\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.086897 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.086837    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-model-cache\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.087096 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.087007    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-tmp-dir\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.088983 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.088935    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b747b84b-0b77-4776-9c2e-e101a00f4c21-tls-certs\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.089295 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.089272    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-dshm\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.146188 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.146158    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87t4v\" (UniqueName: \"kubernetes.io/projected/b747b84b-0b77-4776-9c2e-e101a00f4c21-kube-api-access-87t4v\") pod \"simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") " pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.177769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.177736    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:27.249043 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.249002    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:27.630206 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.630129    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"]
Nov 04 03:32:27.636726 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:32:27.636685    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb747b84b_0b77_4776_9c2e_e101a00f4c21.slice/crio-53ed6749d8613cd3d3fe58c60ce32a349176011eb45e0cfb62ac86a583dc0f6f WatchSource:0}: Error finding container 53ed6749d8613cd3d3fe58c60ce32a349176011eb45e0cfb62ac86a583dc0f6f: Status 404 returned error can't find the container with id 53ed6749d8613cd3d3fe58c60ce32a349176011eb45e0cfb62ac86a583dc0f6f
Nov 04 03:32:27.979092 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.979014    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh" event={"ID":"b747b84b-0b77-4776-9c2e-e101a00f4c21","Type":"ContainerStarted","Data":"6b6ab39e42213c23124f2a3594c196f894a6723383b7e3bd95d5a1c2f87e707b"}
Nov 04 03:32:27.979092 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:27.979058    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh" event={"ID":"b747b84b-0b77-4776-9c2e-e101a00f4c21","Type":"ContainerStarted","Data":"53ed6749d8613cd3d3fe58c60ce32a349176011eb45e0cfb62ac86a583dc0f6f"}
Nov 04 03:32:28.099082 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:28.099031    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh" podStartSLOduration=2.09901329 podStartE2EDuration="2.09901329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:32:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:32:28.098788354 +0000 UTC m=+917.913579630" watchObservedRunningTime="2026-11-04 03:32:28.09901329 +0000 UTC m=+917.913804567"
Nov 04 03:32:28.249744 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:28.249703    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:32.018024 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:32.017989    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:32.648346 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:32:32.648308    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:32:36.909595 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:36.909563    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:37.249284 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:37.249244    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:37.261934 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:37.261911    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:37.759029 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:37.758993    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:38.036713 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:38.036634    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:32:41.903386 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:41.903347    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:42.233750 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:42.233660    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:32:46.722281 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:46.722230    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:48.725113 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.725078    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"]
Nov 04 03:32:48.729900 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.729880    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.788016 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.787987    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-dshm\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.788016 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.788019    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-model-cache\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.788207 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.788048    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1152bcd4-4f6e-4068-827d-02514564797d-tls-certs\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.788207 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.788151    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-home\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.788294 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.788212    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-tmp-dir\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.788294 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.788260    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m276p\" (UniqueName: \"kubernetes.io/projected/1152bcd4-4f6e-4068-827d-02514564797d-kube-api-access-m276p\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.819743 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.819720    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:48.825944 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.825921    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-092f8c\"/\"kube-root-ca.crt\""
Nov 04 03:32:48.825944 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.825921    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-092f8c\"/\"openshift-service-ca.crt\""
Nov 04 03:32:48.826104 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.825978    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-092f8c\"/\"default-dockercfg-lpdkc\""
Nov 04 03:32:48.826104 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.826064    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-092f8c\"/\"simulator-w0-092f8c-kserve-self-signed-certs\""
Nov 04 03:32:48.827682 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.827660    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"]
Nov 04 03:32:48.889067 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.889037    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m276p\" (UniqueName: \"kubernetes.io/projected/1152bcd4-4f6e-4068-827d-02514564797d-kube-api-access-m276p\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.889255 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.889109    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-dshm\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.889255 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.889128    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-model-cache\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.889255 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.889150    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1152bcd4-4f6e-4068-827d-02514564797d-tls-certs\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.889255 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.889166    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-home\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.889471 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.889401    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-tmp-dir\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.889643 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.889609    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-home\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.889764 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.889634    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-model-cache\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.889823 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.889779    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-tmp-dir\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.891347 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.891329    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-dshm\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.891602 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.891587    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1152bcd4-4f6e-4068-827d-02514564797d-tls-certs\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:48.962518 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:48.962481    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m276p\" (UniqueName: \"kubernetes.io/projected/1152bcd4-4f6e-4068-827d-02514564797d-kube-api-access-m276p\") pod \"simulator-w0-092f8c-kserve-5648878f9c-cb7dt\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") " pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:49.042209 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:49.042172    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:49.422178 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:49.422097    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"]
Nov 04 03:32:49.428205 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:32:49.428181    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1152bcd4_4f6e_4068_827d_02514564797d.slice/crio-83c30c27cb255deb90192f2c2b427258356ad873f5f24c4187f03ac42f22ee7a WatchSource:0}: Error finding container 83c30c27cb255deb90192f2c2b427258356ad873f5f24c4187f03ac42f22ee7a: Status 404 returned error can't find the container with id 83c30c27cb255deb90192f2c2b427258356ad873f5f24c4187f03ac42f22ee7a
Nov 04 03:32:50.080919 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:50.080885    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt" event={"ID":"1152bcd4-4f6e-4068-827d-02514564797d","Type":"ContainerStarted","Data":"d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018"}
Nov 04 03:32:50.081309 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:50.080930    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt" event={"ID":"1152bcd4-4f6e-4068-827d-02514564797d","Type":"ContainerStarted","Data":"83c30c27cb255deb90192f2c2b427258356ad873f5f24c4187f03ac42f22ee7a"}
Nov 04 03:32:50.192102 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:50.192045    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt" podStartSLOduration=2.192029222 podStartE2EDuration="2.192029222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:32:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:32:50.190450398 +0000 UTC m=+940.005241676" watchObservedRunningTime="2026-11-04 03:32:50.192029222 +0000 UTC m=+940.006820500"
Nov 04 03:32:51.042588 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:51.042541    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:51.609370 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:51.609324    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:56.674640 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:56.674603    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:32:59.043039 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:59.043005    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:59.055161 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:59.055132    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:32:59.129877 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:32:59.129837    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:33:01.435323 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:01.435274    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:02.649304 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:33:02.649268    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:33:05.111503 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:05.111471    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:06.207528 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:06.207490    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:08.443672 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.443641    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"]
Nov 04 03:33:08.447137 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.447115    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.541027 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.540990    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:08.542663 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.542639    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-fba151\"/\"premium-w2-fba151-kserve-self-signed-certs\""
Nov 04 03:33:08.544232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.544212    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"]
Nov 04 03:33:08.579169 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.579143    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-tmp-dir\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.579169 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.579169    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2c2z\" (UniqueName: \"kubernetes.io/projected/70953f5d-4cbc-4cb5-8136-4045ee3af987-kube-api-access-v2c2z\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.579326 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.579206    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-home\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.579372 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.579357    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70953f5d-4cbc-4cb5-8136-4045ee3af987-tls-certs\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.579420 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.579382    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-dshm\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.579420 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.579402    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-model-cache\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.680685 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.680646    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70953f5d-4cbc-4cb5-8136-4045ee3af987-tls-certs\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.680895 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.680690    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-dshm\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.680895 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.680734    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-model-cache\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.681022 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.680876    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-tmp-dir\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.681088 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.681016    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2c2z\" (UniqueName: \"kubernetes.io/projected/70953f5d-4cbc-4cb5-8136-4045ee3af987-kube-api-access-v2c2z\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.681151 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.681111    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-home\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.681312 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.681287    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-tmp-dir\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.681486 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.681393    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-model-cache\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.681588 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.681570    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-home\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.683116 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.683095    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-dshm\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.683605 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.683589    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70953f5d-4cbc-4cb5-8136-4045ee3af987-tls-certs\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.739042 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.739009    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2c2z\" (UniqueName: \"kubernetes.io/projected/70953f5d-4cbc-4cb5-8136-4045ee3af987-kube-api-access-v2c2z\") pod \"premium-w2-fba151-kserve-575dfdb84d-l5dcf\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") " pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:08.758698 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:08.758664    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:09.132684 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:09.132611    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"]
Nov 04 03:33:09.139324 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:33:09.139287    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70953f5d_4cbc_4cb5_8136_4045ee3af987.slice/crio-c5e32ee05480ae796c0149683c0bbccf39dd9db76fb0053755d8de128f96ef56 WatchSource:0}: Error finding container c5e32ee05480ae796c0149683c0bbccf39dd9db76fb0053755d8de128f96ef56: Status 404 returned error can't find the container with id c5e32ee05480ae796c0149683c0bbccf39dd9db76fb0053755d8de128f96ef56
Nov 04 03:33:09.165426 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:09.165397    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf" event={"ID":"70953f5d-4cbc-4cb5-8136-4045ee3af987","Type":"ContainerStarted","Data":"c5e32ee05480ae796c0149683c0bbccf39dd9db76fb0053755d8de128f96ef56"}
Nov 04 03:33:10.093004 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:10.092969    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:33:10.171208 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:10.171174    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf" event={"ID":"70953f5d-4cbc-4cb5-8136-4045ee3af987","Type":"ContainerStarted","Data":"81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f"}
Nov 04 03:33:10.281231 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:10.281175    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf" podStartSLOduration=2.281158785 podStartE2EDuration="2.281158785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:33:10.280804139 +0000 UTC m=+960.095595416" watchObservedRunningTime="2026-11-04 03:33:10.281158785 +0000 UTC m=+960.095950063"
Nov 04 03:33:10.759610 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:10.759576    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:11.107267 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:11.107191    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:15.906332 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:15.906294    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:16.236613 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:16.236530    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:18.759322 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:18.759284    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:18.771497 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:18.771470    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:19.222435 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:19.222363    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:33:20.735108 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:20.735063    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:22.290176 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:22.290142    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:25.611992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:25.611951    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:27.452727 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:27.452690    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:28.156194 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:28.156162    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:33:30.510667 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:30.510625    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:32.648740 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:33:32.648704    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:33:35.323239 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:35.323203    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:37.316463 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:37.316436    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:40.217737 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:40.217699    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:45.016733 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:45.016694    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:49.248028 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:49.247989    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:49.928910 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:49.928881    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:54.907269 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:54.907237    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:33:56.280679 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:56.280638    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:33:59.723279 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:33:59.723236    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:02.648325 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:34:02.648283    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:34:03.871629 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.871598    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"]
Nov 04 03:34:03.875348 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.875326    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.885964 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.885944    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-model-cache\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.886046 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.885988    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-dshm\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.886046 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.886005    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-home\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.886161 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.886117    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-tmp-dir\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.886215 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.886203    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5163cb68-1d76-43af-8654-990a89eb7405-tls-certs\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.886255 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.886243    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j5h5\" (UniqueName: \"kubernetes.io/projected/5163cb68-1d76-43af-8654-990a89eb7405-kube-api-access-6j5h5\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.964618 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.964587    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:03.970887 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.970868    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-092f8c\"/\"premium-w0-092f8c-kserve-self-signed-certs\""
Nov 04 03:34:03.972566 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.972544    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"]
Nov 04 03:34:03.987179 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.987153    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5163cb68-1d76-43af-8654-990a89eb7405-tls-certs\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.987290 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.987205    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j5h5\" (UniqueName: \"kubernetes.io/projected/5163cb68-1d76-43af-8654-990a89eb7405-kube-api-access-6j5h5\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.987290 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.987246    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-model-cache\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.987290 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.987282    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-dshm\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.987449 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.987305    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-home\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.987449 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.987340    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-tmp-dir\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.987727 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.987695    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-model-cache\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.987794 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.987715    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-home\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.987794 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.987764    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-tmp-dir\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.989601 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.989574    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-dshm\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:03.989699 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:03.989638    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5163cb68-1d76-43af-8654-990a89eb7405-tls-certs\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:04.046820 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:04.046790    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j5h5\" (UniqueName: \"kubernetes.io/projected/5163cb68-1d76-43af-8654-990a89eb7405-kube-api-access-6j5h5\") pod \"premium-w0-092f8c-kserve-5bc544cf6-kpqrg\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") " pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:04.187458 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:04.187355    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:04.571669 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:04.571634    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"]
Nov 04 03:34:04.578575 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:34:04.578545    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5163cb68_1d76_43af_8654_990a89eb7405.slice/crio-7ed8dc31444df0a5c84186868cbef7c7be0b97ac4ade26f3ec582b5d3dc52c16 WatchSource:0}: Error finding container 7ed8dc31444df0a5c84186868cbef7c7be0b97ac4ade26f3ec582b5d3dc52c16: Status 404 returned error can't find the container with id 7ed8dc31444df0a5c84186868cbef7c7be0b97ac4ade26f3ec582b5d3dc52c16
Nov 04 03:34:04.629558 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:04.629532    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:05.435624 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:05.435586    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg" event={"ID":"5163cb68-1d76-43af-8654-990a89eb7405","Type":"ContainerStarted","Data":"50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277"}
Nov 04 03:34:05.436028 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:05.435636    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg" event={"ID":"5163cb68-1d76-43af-8654-990a89eb7405","Type":"ContainerStarted","Data":"7ed8dc31444df0a5c84186868cbef7c7be0b97ac4ade26f3ec582b5d3dc52c16"}
Nov 04 03:34:05.462411 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:05.462376    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:05.545397 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:05.545346    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg" podStartSLOduration=2.545332419 podStartE2EDuration="2.545332419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:34:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:34:05.544497332 +0000 UTC m=+1015.359288609" watchObservedRunningTime="2026-11-04 03:34:05.545332419 +0000 UTC m=+1015.360123695"
Nov 04 03:34:06.188254 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:06.188224    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:09.618032 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:09.617992    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:14.187514 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:14.187475    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:14.200324 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:14.200293    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:14.490180 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:14.490153    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:34:14.551620 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:14.551586    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:19.510672 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:19.510630    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:24.438198 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:24.438111    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:27.267415 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:27.267376    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:34:29.400698 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:29.400658    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:29.888728 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:29.888696    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:31.743997 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.743961    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"]
Nov 04 03:34:31.747589 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.747568    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.797017 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.796989    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-092f8c-openshift-default-dockercfg-9bd4m\""
Nov 04 03:34:31.845342 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.845310    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"]
Nov 04 03:34:31.859326 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.859300    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.859464 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.859343    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.859464 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.859370    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.859464 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.859429    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.859582 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.859544    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.859582 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.859568    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t9q2\" (UniqueName: \"kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-kube-api-access-7t9q2\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.859662 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.859642    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.859749 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.859732    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.859837 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.859820    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.960590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.960561    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.960590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.960595    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.960776 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.960624    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.960776 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.960733    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.960911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.960790    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.960911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.960819    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.960911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.960888    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.961078 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.960918    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t9q2\" (UniqueName: \"kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-kube-api-access-7t9q2\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.961078 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.960956    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.961078 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.961011    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-credential-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.961220 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.961133    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-data\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.961287 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.961269    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-socket\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.961367 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.961347    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istiod-ca-cert\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.961418 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.961369    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-certs\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.963143 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.963123    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-envoy\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:31.963258 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:31.963240    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-podinfo\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:32.021943 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:32.021874    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-token\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:32.022070 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:32.021986    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t9q2\" (UniqueName: \"kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-kube-api-access-7t9q2\") pod \"e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") " pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:32.059489 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:32.059460    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:32.484038 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:32.483957    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"]
Nov 04 03:34:32.489646 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:34:32.489606    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1e4ba6_3c59_4786_ba2e_ffb5d0402924.slice/crio-3a032f3c021331fd37f4229470a80f52cd07342dd733592a1529de136e460741 WatchSource:0}: Error finding container 3a032f3c021331fd37f4229470a80f52cd07342dd733592a1529de136e460741: Status 404 returned error can't find the container with id 3a032f3c021331fd37f4229470a80f52cd07342dd733592a1529de136e460741
Nov 04 03:34:32.492010 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:32.491975    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:34:32.492129 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:32.492053    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:34:32.492129 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:32.492095    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:34:32.599444 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:32.599410    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47" event={"ID":"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924","Type":"ContainerStarted","Data":"09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527"}
Nov 04 03:34:32.599651 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:32.599470    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47" event={"ID":"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924","Type":"ContainerStarted","Data":"3a032f3c021331fd37f4229470a80f52cd07342dd733592a1529de136e460741"}
Nov 04 03:34:32.652434 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:34:32.652396    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:34:32.715549 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:32.715498    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47" podStartSLOduration=1.715483172 podStartE2EDuration="1.715483172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:34:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:34:32.71432894 +0000 UTC m=+1042.529120216" watchObservedRunningTime="2026-11-04 03:34:32.715483172 +0000 UTC m=+1042.530274450"
Nov 04 03:34:33.062181 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:33.060246    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:33.063028 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:33.062998    4420 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Nov 04 03:34:33.063251 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:33.063224    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47" podUID="fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Nov 04 03:34:34.064195 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:34.064160    4420 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Nov 04 03:34:34.064769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:34.064738    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47" podUID="fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Nov 04 03:34:34.302800 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:34.302723    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:35.068229 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:35.068193    4420 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Nov 04 03:34:35.068877 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:35.068826    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47" podUID="fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Nov 04 03:34:36.064691 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:36.064248    4420 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Nov 04 03:34:36.064691 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:36.064311    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47" podUID="fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Nov 04 03:34:36.500595 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:36.500562    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:36.708693 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:36.708659    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:37.061638 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:37.061193    4420 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Nov 04 03:34:37.061638 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:37.061254    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47" podUID="fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Nov 04 03:34:38.060303 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:38.060264    4420 patch_prober.go:29] interesting pod/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Nov 04 03:34:38.061250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:38.060837    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47" podUID="fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Nov 04 03:34:39.198107 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:39.198069    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:39.220323 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:39.220288    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:39.220753 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:39.220728    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:39.221578 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:39.221558    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:34:42.605509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.605478    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"]
Nov 04 03:34:42.609338 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.609316    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.658748 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.658709    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-ee1947-openshift-default-dockercfg-2xn5j\""
Nov 04 03:34:42.668053 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.668029    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.668170 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.668070    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.668170 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.668099    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.668170 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.668156    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.668272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.668230    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.668272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.668259    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/656d246c-97d1-46e3-881c-a31c4a0eb32e-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.668332 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.668282    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vws9n\" (UniqueName: \"kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-kube-api-access-vws9n\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.668429 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.668409    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.668467 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.668440    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.707628 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.707607    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"]
Nov 04 03:34:42.769343 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.769320    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.769476 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.769348    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.769476 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.769367    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.769476 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.769392    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.769476 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.769420    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.769476 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.769443    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.769753 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.769498    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.769753 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.769540    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/656d246c-97d1-46e3-881c-a31c4a0eb32e-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.769753 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.769570    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vws9n\" (UniqueName: \"kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-kube-api-access-vws9n\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.770153 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.770132    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.770245 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.770181    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.770297 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.770255    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/656d246c-97d1-46e3-881c-a31c4a0eb32e-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.770297 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.770255    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.770377 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.770360    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.772011 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.771993    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.772065 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.772020    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.827397 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.827362    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.827561 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.827423    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vws9n\" (UniqueName: \"kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-kube-api-access-vws9n\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:42.922792 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:42.922712    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:43.350622 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:43.350589    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"]
Nov 04 03:34:43.357073 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:34:43.357046    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod656d246c_97d1_46e3_881c_a31c4a0eb32e.slice/crio-176aecdfff00cb2be42c467df3bab3ad30ca5fdf0b29565a9641900cc9d3cb97 WatchSource:0}: Error finding container 176aecdfff00cb2be42c467df3bab3ad30ca5fdf0b29565a9641900cc9d3cb97: Status 404 returned error can't find the container with id 176aecdfff00cb2be42c467df3bab3ad30ca5fdf0b29565a9641900cc9d3cb97
Nov 04 03:34:43.359263 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:43.359230    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:34:43.359330 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:43.359300    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:34:43.359330 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:43.359326    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:34:43.650016 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:43.649917    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz" event={"ID":"656d246c-97d1-46e3-881c-a31c4a0eb32e","Type":"ContainerStarted","Data":"bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec"}
Nov 04 03:34:43.650016 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:43.649973    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz" event={"ID":"656d246c-97d1-46e3-881c-a31c4a0eb32e","Type":"ContainerStarted","Data":"176aecdfff00cb2be42c467df3bab3ad30ca5fdf0b29565a9641900cc9d3cb97"}
Nov 04 03:34:43.765336 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:43.765282    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz" podStartSLOduration=1.765266491 podStartE2EDuration="1.765266491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:34:43.763715904 +0000 UTC m=+1053.578507180" watchObservedRunningTime="2026-11-04 03:34:43.765266491 +0000 UTC m=+1053.580057768"
Nov 04 03:34:43.923453 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:43.923362    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:43.998436 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:43.998403    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:44.925367 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:44.925336    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Nov 04 03:34:44.925770 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:44.925393    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz" podUID="656d246c-97d1-46e3-881c-a31c4a0eb32e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Nov 04 03:34:45.923679 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:45.923649    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Nov 04 03:34:45.923954 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:45.923702    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz" podUID="656d246c-97d1-46e3-881c-a31c4a0eb32e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Nov 04 03:34:46.923320 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:46.923286    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Nov 04 03:34:46.923711 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:46.923337    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz" podUID="656d246c-97d1-46e3-881c-a31c4a0eb32e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Nov 04 03:34:47.923518 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:47.923477    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Nov 04 03:34:47.924054 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:47.923535    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz" podUID="656d246c-97d1-46e3-881c-a31c4a0eb32e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Nov 04 03:34:48.923998 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:48.923967    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Nov 04 03:34:48.924387 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:48.924016    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz" podUID="656d246c-97d1-46e3-881c-a31c4a0eb32e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Nov 04 03:34:48.926258 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:48.926231    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:50.032956 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:50.032924    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:50.033712 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:50.033687    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:50.034560 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:50.034540    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:34:50.131783 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:50.131749    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"]
Nov 04 03:34:50.132021 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:50.131997    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92" podUID="48072923-8c97-4319-878d-fca3595c4d5c" containerName="istio-proxy" containerID="cri-o://70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d" gracePeriod=30
Nov 04 03:34:53.746925 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:53.746892    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:34:55.386083 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.386056    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:34:55.496945 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.496915    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-credential-socket\") pod \"48072923-8c97-4319-878d-fca3595c4d5c\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") "
Nov 04 03:34:55.497120 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.496959    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48072923-8c97-4319-878d-fca3595c4d5c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48072923-8c97-4319-878d-fca3595c4d5c-istiod-ca-cert\") pod \"48072923-8c97-4319-878d-fca3595c4d5c\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") "
Nov 04 03:34:55.497120 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.496980    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-socket\") pod \"48072923-8c97-4319-878d-fca3595c4d5c\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") "
Nov 04 03:34:55.497120 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497016    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-envoy\") pod \"48072923-8c97-4319-878d-fca3595c4d5c\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") "
Nov 04 03:34:55.497120 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497064    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-data\") pod \"48072923-8c97-4319-878d-fca3595c4d5c\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") "
Nov 04 03:34:55.497120 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497101    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/48072923-8c97-4319-878d-fca3595c4d5c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48072923-8c97-4319-878d-fca3595c4d5c-istio-podinfo\") pod \"48072923-8c97-4319-878d-fca3595c4d5c\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") "
Nov 04 03:34:55.497342 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497132    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-istio-token\" (UniqueName: \"kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-istio-token\") pod \"48072923-8c97-4319-878d-fca3595c4d5c\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") "
Nov 04 03:34:55.497342 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497164    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-kube-api-access-9wdcx\" (UniqueName: \"kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-kube-api-access-9wdcx\") pod \"48072923-8c97-4319-878d-fca3595c4d5c\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") "
Nov 04 03:34:55.497342 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497205    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-certs\") pod \"48072923-8c97-4319-878d-fca3595c4d5c\" (UID: \"48072923-8c97-4319-878d-fca3595c4d5c\") "
Nov 04 03:34:55.497342 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497258    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-credential-socket" pod "48072923-8c97-4319-878d-fca3595c4d5c" (UID: "48072923-8c97-4319-878d-fca3595c4d5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:34:55.497508 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497377    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48072923-8c97-4319-878d-fca3595c4d5c-istiod-ca-cert" pod "48072923-8c97-4319-878d-fca3595c4d5c" (UID: "48072923-8c97-4319-878d-fca3595c4d5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:34:55.497580 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497561    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-socket" pod "48072923-8c97-4319-878d-fca3595c4d5c" (UID: "48072923-8c97-4319-878d-fca3595c4d5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:34:55.497646 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497574    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-certs" pod "48072923-8c97-4319-878d-fca3595c4d5c" (UID: "48072923-8c97-4319-878d-fca3595c4d5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:34:55.497734 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497701    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-data" pod "48072923-8c97-4319-878d-fca3595c4d5c" (UID: "48072923-8c97-4319-878d-fca3595c4d5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:34:55.497891 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497754    4420 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:55.497891 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497776    4420 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-credential-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:55.497891 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497792    4420 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48072923-8c97-4319-878d-fca3595c4d5c-istiod-ca-cert\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:55.497891 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.497805    4420 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-workload-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:55.499562 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.499531    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48072923-8c97-4319-878d-fca3595c4d5c-istio-podinfo" pod "48072923-8c97-4319-878d-fca3595c4d5c" (UID: "48072923-8c97-4319-878d-fca3595c4d5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:34:55.499659 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.499574    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-envoy" pod "48072923-8c97-4319-878d-fca3595c4d5c" (UID: "48072923-8c97-4319-878d-fca3595c4d5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:34:55.499659 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.499630    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-kube-api-access-9wdcx" pod "48072923-8c97-4319-878d-fca3595c4d5c" (UID: "48072923-8c97-4319-878d-fca3595c4d5c"). InnerVolumeSpecName "kube-api-access-9wdcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:34:55.499728 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.499664    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-istio-token" pod "48072923-8c97-4319-878d-fca3595c4d5c" (UID: "48072923-8c97-4319-878d-fca3595c4d5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:34:55.598492 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.598456    4420 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48072923-8c97-4319-878d-fca3595c4d5c-istio-podinfo\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:55.598492 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.598486    4420 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-istio-token\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:55.598492 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.598496    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wdcx\" (UniqueName: \"kubernetes.io/projected/48072923-8c97-4319-878d-fca3595c4d5c-kube-api-access-9wdcx\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:55.598720 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.598507    4420 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-envoy\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:55.598720 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.598515    4420 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48072923-8c97-4319-878d-fca3595c4d5c-istio-data\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:34:55.725939 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.725902    4420 generic.go:356] "Generic (PLEG): container finished" podID="48072923-8c97-4319-878d-fca3595c4d5c" containerID="70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d" exitCode=0
Nov 04 03:34:55.726106 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.725970    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"
Nov 04 03:34:55.726106 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.725992    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92" event={"ID":"48072923-8c97-4319-878d-fca3595c4d5c","Type":"ContainerDied","Data":"70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d"}
Nov 04 03:34:55.726106 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.726077    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92" event={"ID":"48072923-8c97-4319-878d-fca3595c4d5c","Type":"ContainerDied","Data":"69bb3ff17b49356409af9263003c24d1c2563bb84487039772c323cdda99e3fd"}
Nov 04 03:34:55.726106 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.726100    4420 scope.go:122] "RemoveContainer" containerID="70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d"
Nov 04 03:34:55.735500 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.735477    4420 scope.go:122] "RemoveContainer" containerID="70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d"
Nov 04 03:34:55.735794 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:34:55.735767    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d\": container with ID starting with 70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d not found: ID does not exist" containerID="70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d"
Nov 04 03:34:55.735857 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.735807    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d"} err="failed to get container status \"70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d\": rpc error: code = NotFound desc = could not find container \"70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d\": container with ID starting with 70c7eb783681bb4b891c612a9eec6be53d3210304db0e27ac6915ba58db3ba0d not found: ID does not exist"
Nov 04 03:34:55.884453 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.884377    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"]
Nov 04 03:34:55.888253 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:55.888229    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-rjj92"]
Nov 04 03:34:56.306151 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:56.306123    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48072923-8c97-4319-878d-fca3595c4d5c" path="/var/lib/kubelet/pods/48072923-8c97-4319-878d-fca3595c4d5c/volumes"
Nov 04 03:34:58.810240 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:34:58.810204    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:02.649307 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:35:02.649270    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:35:03.610229 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:03.610192    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:05.441139 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:05.441108    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:05.814805 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:05.814766    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:08.533200 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:08.533163    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:12.969463 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:12.969420    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:35:13.332180 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:13.332148    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:18.190806 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:18.190769    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:23.023877 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:23.023830    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:27.926672 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:27.926630    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:32.649284 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:35:32.649246    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:35:32.810709 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:32.810666    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:34.642377 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:34.642344    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:34.790068 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:34.790036    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:37.711511 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:37.711469    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:42.608678 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:42.608635    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:47.424435 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:47.424383    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:50.215418 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:50.215385    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:51.126503 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:51.126461    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:51.333483 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:51.333443    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:52.315489 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:52.315458    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:57.118318 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:57.118281    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:57.981436 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:57.981400    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:58.933615 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:58.933579    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:59.660531 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:59.660498    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:35:59.809298 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:35:59.809262    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:02.010722 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:02.010681    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:02.649394 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:36:02.649356    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:36:03.746717 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:03.746682    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:03.920130 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:03.920096    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:04.073950 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:04.073861    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:05.842940 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:05.842905    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:06.161705 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:06.161605    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:06.324741 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:06.324711    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:06.538436 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:06.538398    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:06.745243 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:06.745205    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:06.895771 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:06.895687    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:07.052542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:07.052497    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:07.205651 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:07.205546    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:07.353836 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:07.353802    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:08.658306 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:08.658270    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:11.886651 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:11.886610    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:12.085897 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:12.085869    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:36:12.090048 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:12.090020    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:12.534760 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:12.534720    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:17.429724 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:17.429689    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:22.323033 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:22.322992    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:24.147097 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.147062    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"]
Nov 04 03:36:24.147502 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.147486    4420 cpu_manager.go:414] "containerMap: removing container" podUID="48072923-8c97-4319-878d-fca3595c4d5c" containerName="istio-proxy"
Nov 04 03:36:24.147545 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.147504    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48072923-8c97-4319-878d-fca3595c4d5c" containerName="istio-proxy"
Nov 04 03:36:24.147594 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.147584    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="48072923-8c97-4319-878d-fca3595c4d5c" containerName="istio-proxy"
Nov 04 03:36:24.151019 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.150997    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.240189 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.240161    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:24.246677 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.246657    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-092f8c\"/\"distinct-w0-092f8c-kserve-self-signed-certs\""
Nov 04 03:36:24.248531 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.248512    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"]
Nov 04 03:36:24.272864 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.272813    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6339e78-6825-4246-a5d2-11831cd454af-tls-certs\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.272971 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.272932    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-tmp-dir\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.272971 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.272962    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-dshm\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.273060 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.273037    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-model-cache\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.273097 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.273075    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-home\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.273140 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.273098    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbdq7\" (UniqueName: \"kubernetes.io/projected/f6339e78-6825-4246-a5d2-11831cd454af-kube-api-access-wbdq7\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.374247 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.374214    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6339e78-6825-4246-a5d2-11831cd454af-tls-certs\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.374421 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.374269    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-tmp-dir\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.374421 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.374295    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-dshm\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.374421 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.374351    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-model-cache\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.374421 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.374381    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-home\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.374421 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.374411    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbdq7\" (UniqueName: \"kubernetes.io/projected/f6339e78-6825-4246-a5d2-11831cd454af-kube-api-access-wbdq7\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.374687 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.374662    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-tmp-dir\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.374794 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.374763    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-home\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.374961 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.374806    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-model-cache\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.376606 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.376584    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-dshm\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.377197 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.377175    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6339e78-6825-4246-a5d2-11831cd454af-tls-certs\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.438237 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.438164    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbdq7\" (UniqueName: \"kubernetes.io/projected/f6339e78-6825-4246-a5d2-11831cd454af-kube-api-access-wbdq7\") pod \"distinct-w0-092f8c-kserve-7696458459-nmm5x\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.462485 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.462453    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:24.849513 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:24.849480    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"]
Nov 04 03:36:24.855862 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:36:24.855819    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6339e78_6825_4246_a5d2_11831cd454af.slice/crio-8bf794205c1b7c0e2c540dcc71b9db8158c78c3a347e9c9a9a110a4a3887fb5f WatchSource:0}: Error finding container 8bf794205c1b7c0e2c540dcc71b9db8158c78c3a347e9c9a9a110a4a3887fb5f: Status 404 returned error can't find the container with id 8bf794205c1b7c0e2c540dcc71b9db8158c78c3a347e9c9a9a110a4a3887fb5f
Nov 04 03:36:25.149015 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:25.148920    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x" event={"ID":"f6339e78-6825-4246-a5d2-11831cd454af","Type":"ContainerStarted","Data":"46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e"}
Nov 04 03:36:25.149015 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:25.148966    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x" event={"ID":"f6339e78-6825-4246-a5d2-11831cd454af","Type":"ContainerStarted","Data":"8bf794205c1b7c0e2c540dcc71b9db8158c78c3a347e9c9a9a110a4a3887fb5f"}
Nov 04 03:36:25.259614 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:25.259566    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x" podStartSLOduration=1.259552959 podStartE2EDuration="1.259552959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:36:25.258340305 +0000 UTC m=+1155.073131583" watchObservedRunningTime="2026-11-04 03:36:25.259552959 +0000 UTC m=+1155.074344284"
Nov 04 03:36:25.462663 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:25.462589    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:27.207522 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:27.207480    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:31.991142 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:31.991105    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:32.147255 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:32.147221    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:32.299396 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:32.299304    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:32.649509 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:36:32.649436    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:36:32.914047 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:32.913971    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:36:34.462929 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:34.462895    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:34.475083 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:34.475057    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:35.208697 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:35.208671    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:36:37.244601 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:37.244562    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:38.044852 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.044821    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"]
Nov 04 03:36:38.048249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.048223    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.106205 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.106169    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9bd2\" (UniqueName: \"kubernetes.io/projected/178304db-3ebd-4993-abd1-b283b7274d8a-kube-api-access-r9bd2\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.106468 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.106432    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-dshm\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.106552 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.106475    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/178304db-3ebd-4993-abd1-b283b7274d8a-tls-certs\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.106603 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.106573    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-home\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.106661 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.106640    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-tmp-dir\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.106752 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.106734    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-model-cache\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.138109 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.138081    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:38.144190 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.144168    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-fba151\"/\"distinct-2-w2-fba151-kserve-self-signed-certs\""
Nov 04 03:36:38.145892 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.145874    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"]
Nov 04 03:36:38.207623 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.207598    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/178304db-3ebd-4993-abd1-b283b7274d8a-tls-certs\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.207744 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.207628    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-home\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.207744 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.207665    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-tmp-dir\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.207744 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.207725    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-model-cache\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.207935 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.207760    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9bd2\" (UniqueName: \"kubernetes.io/projected/178304db-3ebd-4993-abd1-b283b7274d8a-kube-api-access-r9bd2\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.207935 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.207812    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-dshm\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.208097 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.208068    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-home\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.208216 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.208192    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-tmp-dir\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.208260 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.208225    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-model-cache\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.210335 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.210310    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-dshm\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.210430 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.210362    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/178304db-3ebd-4993-abd1-b283b7274d8a-tls-certs\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.265418 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.265385    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9bd2\" (UniqueName: \"kubernetes.io/projected/178304db-3ebd-4993-abd1-b283b7274d8a-kube-api-access-r9bd2\") pod \"distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") " pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.360406 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.360320    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:38.735514 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:38.735441    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"]
Nov 04 03:36:38.741641 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:36:38.741611    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod178304db_3ebd_4993_abd1_b283b7274d8a.slice/crio-e4c2ae9cc96600f9584a10f7382b17461b4efd9a06947ac685b49b0149478f08 WatchSource:0}: Error finding container e4c2ae9cc96600f9584a10f7382b17461b4efd9a06947ac685b49b0149478f08: Status 404 returned error can't find the container with id e4c2ae9cc96600f9584a10f7382b17461b4efd9a06947ac685b49b0149478f08
Nov 04 03:36:39.219654 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:39.219618    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t" event={"ID":"178304db-3ebd-4993-abd1-b283b7274d8a","Type":"ContainerStarted","Data":"e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605"}
Nov 04 03:36:39.219831 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:39.219665    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t" event={"ID":"178304db-3ebd-4993-abd1-b283b7274d8a","Type":"ContainerStarted","Data":"e4c2ae9cc96600f9584a10f7382b17461b4efd9a06947ac685b49b0149478f08"}
Nov 04 03:36:39.330701 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:39.330647    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t" podStartSLOduration=1.330634037 podStartE2EDuration="1.330634037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:36:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:36:39.32926338 +0000 UTC m=+1169.144054659" watchObservedRunningTime="2026-11-04 03:36:39.330634037 +0000 UTC m=+1169.145425313"
Nov 04 03:36:39.361370 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:39.361342    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:42.059331 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:42.059300    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:42.261976 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:42.261925    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:42.640034 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:42.639994    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:47.168931 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:47.168885    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:48.360966 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:48.360934    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:48.373374 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:48.373354    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:49.277718 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:49.277691    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:36:52.005379 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:52.005346    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:36:56.902641 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:36:56.902602    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:01.810929 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:01.810892    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:02.648545 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:37:02.648513    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:37:04.979459 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:04.979385    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:37:06.084413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.084382    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"]
Nov 04 03:37:06.088080 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.088062    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.137475 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.137447    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-fba151-openshift-default-dockercfg-v44lg\""
Nov 04 03:37:06.179910 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.179885    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-data\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.180022 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.179928    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-token\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.180022 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.179951    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istiod-ca-cert\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.180114 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.180052    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-envoy\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.180114 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.180085    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgrhs\" (UniqueName: \"kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-kube-api-access-qgrhs\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.180114 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.180107    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-credential-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.180245 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.180138    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-podinfo\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.180245 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.180238    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-certs\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.180311 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.180269    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.187530 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.187506    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"]
Nov 04 03:37:06.235616 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.235589    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"]
Nov 04 03:37:06.235795 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.235776    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" podUID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" containerName="istio-proxy" containerID="cri-o://000d1e1eeb20e91bb047634fa8c3aa0a63adbd84f8cb6842ea293fde0ff173ba" gracePeriod=30
Nov 04 03:37:06.281061 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281031    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-podinfo\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.281197 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281074    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-certs\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.281197 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281100    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.281197 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281182    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-data\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.281362 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281222    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-token\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.281362 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281253    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istiod-ca-cert\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.281362 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281287    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-envoy\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.281362 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281319    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgrhs\" (UniqueName: \"kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-kube-api-access-qgrhs\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.281362 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281353    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-credential-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.281628 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281502    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-certs\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.281628 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281548    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.281733 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281622    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-data\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.281733 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.281706    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-credential-socket\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.282033 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.282007    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istiod-ca-cert\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.283513 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.283495    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-podinfo\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.283734 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.283716    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-envoy\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.338730 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.338669    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-token\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.339035 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.338974    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgrhs\" (UniqueName: \"kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-kube-api-access-qgrhs\") pod \"e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") " pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.400798 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.400773    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:06.518957 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.518918    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:06.822331 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.822299    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"]
Nov 04 03:37:06.828760 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:37:06.828733    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c75e12d_2787_43af_a6f2_1e678b9c33f3.slice/crio-36c7c8c3eabe8e4a1fd8ffb75c35e8604edc5a1f8f1e1c9ab746d7f61ac998c2 WatchSource:0}: Error finding container 36c7c8c3eabe8e4a1fd8ffb75c35e8604edc5a1f8f1e1c9ab746d7f61ac998c2: Status 404 returned error can't find the container with id 36c7c8c3eabe8e4a1fd8ffb75c35e8604edc5a1f8f1e1c9ab746d7f61ac998c2
Nov 04 03:37:06.831035 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.831006    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:37:06.831108 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.831066    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:37:06.831108 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:06.831094    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:37:07.371576 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:07.371532    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" event={"ID":"6c75e12d-2787-43af-a6f2-1e678b9c33f3","Type":"ContainerStarted","Data":"3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342"}
Nov 04 03:37:07.372011 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:07.371585    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" event={"ID":"6c75e12d-2787-43af-a6f2-1e678b9c33f3","Type":"ContainerStarted","Data":"36c7c8c3eabe8e4a1fd8ffb75c35e8604edc5a1f8f1e1c9ab746d7f61ac998c2"}
Nov 04 03:37:07.401123 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:07.401091    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:07.402504 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:07.402484    4420 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 04 03:37:07.402566 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:07.402526    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" podUID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 04 03:37:07.485688 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:07.485620    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" podStartSLOduration=1.485593476 podStartE2EDuration="1.485593476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:37:07.484403306 +0000 UTC m=+1197.299194579" watchObservedRunningTime="2026-11-04 03:37:07.485593476 +0000 UTC m=+1197.300384754"
Nov 04 03:37:07.959397 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:07.959359    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:08.401215 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:08.401182    4420 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 04 03:37:08.401686 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:08.401247    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" podUID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 04 03:37:09.401905 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:09.401875    4420 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 04 03:37:09.402272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:09.401935    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" podUID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 04 03:37:10.401946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:10.401913    4420 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 04 03:37:10.402354 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:10.401960    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" podUID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 04 03:37:10.416452 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:10.416431    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:37:10.416590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:10.416486    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:37:11.395008 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.394905    4420 generic.go:356] "Generic (PLEG): container finished" podID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" containerID="000d1e1eeb20e91bb047634fa8c3aa0a63adbd84f8cb6842ea293fde0ff173ba" exitCode=0
Nov 04 03:37:11.395008 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.394940    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" event={"ID":"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce","Type":"ContainerDied","Data":"000d1e1eeb20e91bb047634fa8c3aa0a63adbd84f8cb6842ea293fde0ff173ba"}
Nov 04 03:37:11.401393 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.401370    4420 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 04 03:37:11.401536 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.401413    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" podUID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 04 03:37:11.530182 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.530125    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:37:11.534152 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.534132    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-podinfo\") pod \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") "
Nov 04 03:37:11.534239 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.534175    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-data\") pod \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") "
Nov 04 03:37:11.534875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.534550    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-kube-api-access-j4lfb\" (UniqueName: \"kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-kube-api-access-j4lfb\") pod \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") "
Nov 04 03:37:11.534875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.534623    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-credential-socket\") pod \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") "
Nov 04 03:37:11.534875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.534755    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-data" pod "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" (UID: "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:37:11.535162 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.534902    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-credential-socket" pod "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" (UID: "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:37:11.535734 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.535714    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-envoy\") pod \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") "
Nov 04 03:37:11.535930 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.535911    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-certs\") pod \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") "
Nov 04 03:37:11.536073 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.536059    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-token\" (UniqueName: \"kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-token\") pod \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") "
Nov 04 03:37:11.536203 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.536188    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istiod-ca-cert\") pod \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") "
Nov 04 03:37:11.536312 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.536298    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-socket\") pod \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\" (UID: \"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce\") "
Nov 04 03:37:11.537049 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.537021    4420 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-data\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:37:11.537173 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.537158    4420 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-credential-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:37:11.537768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.537336    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-kube-api-access-j4lfb" pod "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" (UID: "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce"). InnerVolumeSpecName "kube-api-access-j4lfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:37:11.537768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.537438    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-socket" pod "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" (UID: "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:37:11.537768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.537454    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-podinfo" pod "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" (UID: "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:37:11.537977 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.537913    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-certs" pod "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" (UID: "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:37:11.538200 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.538181    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istiod-ca-cert" pod "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" (UID: "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:37:11.573335 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.572039    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-token" pod "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" (UID: "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:37:11.573335 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.572265    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-envoy" pod "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" (UID: "c9867a80-2334-4de4-b9d0-3ac27eeaa4ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:37:11.614400 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.614368    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:11.638156 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.638133    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4lfb\" (UniqueName: \"kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-kube-api-access-j4lfb\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:37:11.638156 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.638155    4420 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-envoy\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:37:11.638279 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.638166    4420 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:37:11.638279 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.638175    4420 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-token\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:37:11.638279 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.638184    4420 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istiod-ca-cert\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:37:11.638279 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.638192    4420 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-workload-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:37:11.638279 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:11.638200    4420 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce-istio-podinfo\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:37:12.400368 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:12.400335    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q" event={"ID":"c9867a80-2334-4de4-b9d0-3ac27eeaa4ce","Type":"ContainerDied","Data":"13bee1124f900ffc7d00455a2b6f2af9eb6099019a8419b0e1575e1164ea51c2"}
Nov 04 03:37:12.400368 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:12.400366    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"
Nov 04 03:37:12.400624 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:12.400406    4420 scope.go:122] "RemoveContainer" containerID="000d1e1eeb20e91bb047634fa8c3aa0a63adbd84f8cb6842ea293fde0ff173ba"
Nov 04 03:37:12.401172 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:12.401147    4420 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 04 03:37:12.401304 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:12.401200    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" podUID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 04 03:37:12.571647 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:12.571566    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"]
Nov 04 03:37:12.584476 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:12.584450    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-smv2q"]
Nov 04 03:37:13.401171 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:13.401132    4420 patch_prober.go:29] interesting pod/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 04 03:37:13.401347 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:13.401193    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" podUID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 04 03:37:14.306301 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:14.306264    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" path="/var/lib/kubelet/pods/c9867a80-2334-4de4-b9d0-3ac27eeaa4ce/volumes"
Nov 04 03:37:14.405342 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:14.405314    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:14.405665 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:14.405646    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:14.406356 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:14.406338    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:37:16.276491 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:16.276461    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:16.567425 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:16.567337    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:21.510604 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:21.510532    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:22.221721 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:22.221677    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:22.736373 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:22.736337    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:22.893604 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:22.893565    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:26.953299 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:26.953261    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:27.350190 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:27.350144    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:27.593311 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:27.593279    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:27.761712 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:27.761679    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:27.909342 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:27.909303    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:28.176781 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:28.176689    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:29.932037 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:29.932002    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:32.624642 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:32.624605    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:32.649896 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:37:32.649860    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:37:32.772126 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:32.772097    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:37:32.824732 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:32.824698    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:33.286420 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:33.286388    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:33.463360 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:33.463332    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:33.610940 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:33.610865    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:34.249268 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:34.249234    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:34.442327 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:34.442295    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:35.399450 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:35.399415    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:37.561707 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:37.561675    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:37.742256 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:37.742224    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:37.918173 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:37.918098    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:38.113391 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.113356    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"]
Nov 04 03:37:38.113768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.113755    4420 cpu_manager.go:414] "containerMap: removing container" podUID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" containerName="istio-proxy"
Nov 04 03:37:38.113827 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.113769    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" containerName="istio-proxy"
Nov 04 03:37:38.113889 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.113854    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9867a80-2334-4de4-b9d0-3ac27eeaa4ce" containerName="istio-proxy"
Nov 04 03:37:38.117097 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.117076    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"
Nov 04 03:37:38.212745 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.212673    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-6d170d-dockercfg-8fssf\""
Nov 04 03:37:38.212745 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.212687    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-6d170d\""
Nov 04 03:37:38.214640 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.214621    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"]
Nov 04 03:37:38.217130 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.217113    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd3b4c8-659f-49b2-99f8-13355a4ede22-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t\" (UID: \"4dd3b4c8-659f-49b2-99f8-13355a4ede22\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"
Nov 04 03:37:38.217179 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.217166    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjlvn\" (UniqueName: \"kubernetes.io/projected/4dd3b4c8-659f-49b2-99f8-13355a4ede22-kube-api-access-kjlvn\") pod \"payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t\" (UID: \"4dd3b4c8-659f-49b2-99f8-13355a4ede22\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"
Nov 04 03:37:38.317797 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.317773    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjlvn\" (UniqueName: \"kubernetes.io/projected/4dd3b4c8-659f-49b2-99f8-13355a4ede22-kube-api-access-kjlvn\") pod \"payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t\" (UID: \"4dd3b4c8-659f-49b2-99f8-13355a4ede22\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"
Nov 04 03:37:38.318010 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.317953    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd3b4c8-659f-49b2-99f8-13355a4ede22-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t\" (UID: \"4dd3b4c8-659f-49b2-99f8-13355a4ede22\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"
Nov 04 03:37:38.318582 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.318565    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd3b4c8-659f-49b2-99f8-13355a4ede22-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t\" (UID: \"4dd3b4c8-659f-49b2-99f8-13355a4ede22\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"
Nov 04 03:37:38.375612 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.375589    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjlvn\" (UniqueName: \"kubernetes.io/projected/4dd3b4c8-659f-49b2-99f8-13355a4ede22-kube-api-access-kjlvn\") pod \"payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t\" (UID: \"4dd3b4c8-659f-49b2-99f8-13355a4ede22\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"
Nov 04 03:37:38.429331 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.429308    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"
Nov 04 03:37:38.796209 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.796177    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"]
Nov 04 03:37:38.802768 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:37:38.802740    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd3b4c8_659f_49b2_99f8_13355a4ede22.slice/crio-9b45d7213c1f7d1f3cc4d80b7b8929f77d9c5cbde3262612e56c72d08cb085e8 WatchSource:0}: Error finding container 9b45d7213c1f7d1f3cc4d80b7b8929f77d9c5cbde3262612e56c72d08cb085e8: Status 404 returned error can't find the container with id 9b45d7213c1f7d1f3cc4d80b7b8929f77d9c5cbde3262612e56c72d08cb085e8
Nov 04 03:37:38.808002 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:38.807978    4420 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:37:39.551814 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:39.551784    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t" event={"ID":"4dd3b4c8-659f-49b2-99f8-13355a4ede22","Type":"ContainerStarted","Data":"9b45d7213c1f7d1f3cc4d80b7b8929f77d9c5cbde3262612e56c72d08cb085e8"}
Nov 04 03:37:42.566628 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:42.566597    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t" event={"ID":"4dd3b4c8-659f-49b2-99f8-13355a4ede22","Type":"ContainerStarted","Data":"707f5b5ad54487e09c22973d88cea216bd2ed3442b5f9efe13c5ff25d0cbc2fb"}
Nov 04 03:37:42.567063 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:42.566662    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"
Nov 04 03:37:42.676065 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:42.676018    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t" podStartSLOduration=1.850305696 podStartE2EDuration="4.676004185s" totalImagesPullingTime="2.825698489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:37:42.67491752 +0000 UTC m=+1232.489708796" watchObservedRunningTime="2026-11-04 03:37:42.676004185 +0000 UTC m=+1232.490795461"
Nov 04 03:37:42.800763 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:42.800730    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:47.715108 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:47.715076    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:48.790166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:48.790133    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:49.004855 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:49.004807    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:50.881979 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:50.881944    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:52.023595 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:52.023558    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:52.201465 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:52.201428    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:52.957696 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:52.957660    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:53.575586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:53.575551    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"
Nov 04 03:37:57.158727 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.158691    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"]
Nov 04 03:37:57.163118 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.163091    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.201196 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.201166    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-home\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.201342 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.201215    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-tmp-dir\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.201342 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.201239    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqgwf\" (UniqueName: \"kubernetes.io/projected/3b12d15f-129b-4121-85ac-c83c3581fcc2-kube-api-access-mqgwf\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.201436 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.201379    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-dshm\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.201509 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.201488    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-model-cache\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.201570 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.201533    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b12d15f-129b-4121-85ac-c83c3581fcc2-tls-certs\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.259648 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.259615    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-092f8c\"/\"distinct-2-w0-092f8c-kserve-self-signed-certs\""
Nov 04 03:37:57.261514 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.261490    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"]
Nov 04 03:37:57.302783 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.302751    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-home\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.302970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.302801    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-tmp-dir\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.302970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.302821    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqgwf\" (UniqueName: \"kubernetes.io/projected/3b12d15f-129b-4121-85ac-c83c3581fcc2-kube-api-access-mqgwf\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.302970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.302886    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-dshm\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.302970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.302925    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-model-cache\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.302970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.302954    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b12d15f-129b-4121-85ac-c83c3581fcc2-tls-certs\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.303279 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.303254    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-tmp-dir\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.303357 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.303336    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-home\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.303407 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.303358    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-model-cache\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.305342 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.305315    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-dshm\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.305606 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.305588    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b12d15f-129b-4121-85ac-c83c3581fcc2-tls-certs\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.326297 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.326259    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:57.360697 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.360676    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqgwf\" (UniqueName: \"kubernetes.io/projected/3b12d15f-129b-4121-85ac-c83c3581fcc2-kube-api-access-mqgwf\") pod \"distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") " pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.477554 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.477465    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:57.824114 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.824081    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:37:57.881085 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:57.881051    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"]
Nov 04 03:37:57.891103 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:37:57.891073    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b12d15f_129b_4121_85ac_c83c3581fcc2.slice/crio-387ee13811f28d1836dfd08af4be3151a49478f7738a6cc9db5af585bf9a50e3 WatchSource:0}: Error finding container 387ee13811f28d1836dfd08af4be3151a49478f7738a6cc9db5af585bf9a50e3: Status 404 returned error can't find the container with id 387ee13811f28d1836dfd08af4be3151a49478f7738a6cc9db5af585bf9a50e3
Nov 04 03:37:58.644267 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:58.644229    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5" event={"ID":"3b12d15f-129b-4121-85ac-c83c3581fcc2","Type":"ContainerStarted","Data":"fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3"}
Nov 04 03:37:58.644267 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:58.644273    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5" event={"ID":"3b12d15f-129b-4121-85ac-c83c3581fcc2","Type":"ContainerStarted","Data":"387ee13811f28d1836dfd08af4be3151a49478f7738a6cc9db5af585bf9a50e3"}
Nov 04 03:37:58.755622 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:58.755572    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5" podStartSLOduration=1.755558931 podStartE2EDuration="1.755558931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:37:58.753762366 +0000 UTC m=+1248.568553666" watchObservedRunningTime="2026-11-04 03:37:58.755558931 +0000 UTC m=+1248.570350207"
Nov 04 03:37:59.194987 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.194957    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"]
Nov 04 03:37:59.198517 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.198493    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.241552 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.241518    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"]
Nov 04 03:37:59.245275 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.245257    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.296327 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.296298    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"]
Nov 04 03:37:59.324503 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324473    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.324642 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324516    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.324642 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324541    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.324642 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324632    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.324743 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324656    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.324743 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324674    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn7rm\" (UniqueName: \"kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-kube-api-access-jn7rm\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.324743 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324731    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.324861 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324790    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbb2x\" (UniqueName: \"kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-kube-api-access-vbb2x\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.324861 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324809    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.324861 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324834    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.324968 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324895    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.324968 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324923    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.325033 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.324994    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.325066 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.325032    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.325099 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.325074    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e107aafd-b91e-4a9f-8512-66744aec6a95-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.325132 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.325100    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.325132 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.325117    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/114cc480-e80a-41d7-bd80-ce73ceebc155-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.325216 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.325199    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.353107 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.353060    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"]
Nov 04 03:37:59.426739 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.426704    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.426920 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.426750    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.426920 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.426778    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e107aafd-b91e-4a9f-8512-66744aec6a95-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.426920 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.426867    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.426920 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.426899    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/114cc480-e80a-41d7-bd80-ce73ceebc155-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.427150 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.426960    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.427150 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427015    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.427150 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427058    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.427150 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427086    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.427150 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427117    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.427405 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427152    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.427405 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427177    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn7rm\" (UniqueName: \"kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-kube-api-access-jn7rm\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.427405 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427217    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.427405 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427245    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.427405 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427355    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.427405 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427291    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.427722 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427545    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.427722 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427635    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-certs\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.427722 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427709    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e107aafd-b91e-4a9f-8512-66744aec6a95-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.427893 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427730    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/114cc480-e80a-41d7-bd80-ce73ceebc155-istiod-ca-cert\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.427893 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427257    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbb2x\" (UniqueName: \"kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-kube-api-access-vbb2x\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.427893 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427839    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.427893 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427859    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.428098 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427913    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.428098 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427952    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.428098 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.427992    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.428275 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.428250    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-data\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.428334 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.428300    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-credential-socket\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.429716 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.429688    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.429924 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.429907    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.430495 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.430470    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-envoy\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.430651 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.430637    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-podinfo\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.478682 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.478601    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:37:59.485886 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.485012    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.485886 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.485256    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn7rm\" (UniqueName: \"kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-kube-api-access-jn7rm\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.486199 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.486170    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-token\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.486493 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.486476    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbb2x\" (UniqueName: \"kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-kube-api-access-vbb2x\") pod \"e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") " pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.512784 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.512757    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:37:59.558412 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.558376    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:37:59.941676 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.941648    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"]
Nov 04 03:37:59.944664 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.944639    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"]
Nov 04 03:37:59.947607 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:37:59.947583    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode107aafd_b91e_4a9f_8512_66744aec6a95.slice/crio-9c367ea008e2014bed02a61041f8b30704bfb4b547b6c9331f2d19d2c1026227 WatchSource:0}: Error finding container 9c367ea008e2014bed02a61041f8b30704bfb4b547b6c9331f2d19d2c1026227: Status 404 returned error can't find the container with id 9c367ea008e2014bed02a61041f8b30704bfb4b547b6c9331f2d19d2c1026227
Nov 04 03:37:59.950001 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.949969    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:37:59.950071 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.950029    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:37:59.950071 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.950056    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:37:59.987297 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:37:59.987269    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114cc480_e80a_41d7_bd80_ce73ceebc155.slice/crio-a1f0e4b90ab9cceb8ad5125b666f312a857c4f571f798488a955f1f62ca00ebe WatchSource:0}: Error finding container a1f0e4b90ab9cceb8ad5125b666f312a857c4f571f798488a955f1f62ca00ebe: Status 404 returned error can't find the container with id a1f0e4b90ab9cceb8ad5125b666f312a857c4f571f798488a955f1f62ca00ebe
Nov 04 03:37:59.989634 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.989598    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:37:59.989735 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.989682    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:37:59.989735 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:37:59.989726    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:38:00.698377 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:00.698300    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" event={"ID":"114cc480-e80a-41d7-bd80-ce73ceebc155","Type":"ContainerStarted","Data":"70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70"}
Nov 04 03:38:00.698377 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:00.698373    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" event={"ID":"114cc480-e80a-41d7-bd80-ce73ceebc155","Type":"ContainerStarted","Data":"a1f0e4b90ab9cceb8ad5125b666f312a857c4f571f798488a955f1f62ca00ebe"}
Nov 04 03:38:00.701113 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:00.701086    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" event={"ID":"e107aafd-b91e-4a9f-8512-66744aec6a95","Type":"ContainerStarted","Data":"65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4"}
Nov 04 03:38:00.701210 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:00.701138    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" event={"ID":"e107aafd-b91e-4a9f-8512-66744aec6a95","Type":"ContainerStarted","Data":"9c367ea008e2014bed02a61041f8b30704bfb4b547b6c9331f2d19d2c1026227"}
Nov 04 03:38:00.833812 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:00.833748    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" podStartSLOduration=1.833733343 podStartE2EDuration="1.833733343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:00.832328839 +0000 UTC m=+1250.647120114" watchObservedRunningTime="2026-11-04 03:38:00.833733343 +0000 UTC m=+1250.648524624"
Nov 04 03:38:01.513458 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:01.513423    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:38:01.514956 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:01.514932    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 04 03:38:01.515106 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:01.514989    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" podUID="e107aafd-b91e-4a9f-8512-66744aec6a95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 04 03:38:01.564868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:01.560396    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:38:01.564868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:01.560564    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 04 03:38:01.564868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:01.560607    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" podUID="114cc480-e80a-41d7-bd80-ce73ceebc155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 04 03:38:02.513492 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:02.513457    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 04 03:38:02.514011 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:02.513520    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" podUID="e107aafd-b91e-4a9f-8512-66744aec6a95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 04 03:38:02.564867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:02.564244    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 04 03:38:02.564867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:02.564304    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" podUID="114cc480-e80a-41d7-bd80-ce73ceebc155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 04 03:38:02.696444 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:38:02.696410    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:38:02.923933 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:02.923853    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:03.513589 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:03.513557    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 04 03:38:03.514020 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:03.513605    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" podUID="e107aafd-b91e-4a9f-8512-66744aec6a95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 04 03:38:03.563897 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:03.561197    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 04 03:38:03.563897 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:03.561258    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" podUID="114cc480-e80a-41d7-bd80-ce73ceebc155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 04 03:38:04.513480 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:04.513444    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 04 03:38:04.513646 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:04.513489    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" podUID="e107aafd-b91e-4a9f-8512-66744aec6a95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 04 03:38:04.563867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:04.562142    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 04 03:38:04.563867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:04.562197    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" podUID="114cc480-e80a-41d7-bd80-ce73ceebc155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 04 03:38:05.513831 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:05.513797    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 04 03:38:05.514333 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:05.513880    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" podUID="e107aafd-b91e-4a9f-8512-66744aec6a95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 04 03:38:05.562866 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:05.559135    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 04 03:38:05.562866 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:05.559187    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" podUID="114cc480-e80a-41d7-bd80-ce73ceebc155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 04 03:38:06.513979 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:06.513940    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 04 03:38:06.514328 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:06.514003    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" podUID="e107aafd-b91e-4a9f-8512-66744aec6a95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 04 03:38:06.559451 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:06.559416    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 04 03:38:06.559589 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:06.559477    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" podUID="114cc480-e80a-41d7-bd80-ce73ceebc155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 04 03:38:06.865944 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:06.865868    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:07.477902 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:07.477871    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:38:07.497433 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:07.497404    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:38:07.517263 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:07.517240    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:38:07.517601 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:07.517519    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:38:07.518290 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:07.518272    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:38:07.562896 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:07.562875    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:38:07.563140 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:07.563124    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:38:07.563713 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:07.563699    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:38:07.607536 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:07.607491    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" podStartSLOduration=8.607476342 podStartE2EDuration="8.607476342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:00.955465583 +0000 UTC m=+1250.770256886" watchObservedRunningTime="2026-11-04 03:38:07.607476342 +0000 UTC m=+1257.422267620"
Nov 04 03:38:07.758974 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:07.758917    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:38:07.880543 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:07.880446    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:08.038682 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:08.038604    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:08.200110 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:08.200062    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:12.899533 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:12.899496    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:13.432578 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:13.432546    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:38:14.905986 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:14.905949    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:15.104161 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:15.104125    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:17.750672 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:17.750642    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:17.948019 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:17.947985    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:21.446685 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:21.446641    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:21.927196 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:21.927164    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"]
Nov 04 03:38:21.927386 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:21.927362    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" podUID="78034d31-2927-4a4d-aefe-85df17ec82c9" containerName="istio-proxy" containerID="cri-o://820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a" gracePeriod=30
Nov 04 03:38:22.373702 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.373653    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:22.597278 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.597243    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"]
Nov 04 03:38:22.604391 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.604368    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.657988 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.657927    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"]
Nov 04 03:38:22.701491 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.701462    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-243e19a17a73\""
Nov 04 03:38:22.701491 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.701467    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Nov 04 03:38:22.701706 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.701495    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Nov 04 03:38:22.701706 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.701498    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Nov 04 03:38:22.701706 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.701550    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Nov 04 03:38:22.701706 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.701604    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-mlxkm\""
Nov 04 03:38:22.715135 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.715114    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-routing-overlay-volume\") pod \"payload-processing-external-model-5bcd5df458-7z9tf\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") " pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.715206 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.715149    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-provider-credentials\") pod \"payload-processing-external-model-5bcd5df458-7z9tf\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") " pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.715206 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.715167    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-plugins-config-volume\") pod \"payload-processing-external-model-5bcd5df458-7z9tf\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") " pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.715303 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.715290    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkqt8\" (UniqueName: \"kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-kube-api-access-qkqt8\") pod \"payload-processing-external-model-5bcd5df458-7z9tf\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") " pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.816694 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.816653    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-routing-overlay-volume\") pod \"payload-processing-external-model-5bcd5df458-7z9tf\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") " pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.816877 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.816714    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-provider-credentials\") pod \"payload-processing-external-model-5bcd5df458-7z9tf\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") " pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.816877 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.816734    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-plugins-config-volume\") pod \"payload-processing-external-model-5bcd5df458-7z9tf\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") " pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.816877 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.816783    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkqt8\" (UniqueName: \"kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-kube-api-access-qkqt8\") pod \"payload-processing-external-model-5bcd5df458-7z9tf\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") " pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.817436 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.817414    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-routing-overlay-volume\") pod \"payload-processing-external-model-5bcd5df458-7z9tf\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") " pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.817562 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.817544    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-plugins-config-volume\") pod \"payload-processing-external-model-5bcd5df458-7z9tf\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") " pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.819227 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.819209    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-provider-credentials\") pod \"payload-processing-external-model-5bcd5df458-7z9tf\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") " pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.880999 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.880960    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkqt8\" (UniqueName: \"kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-kube-api-access-qkqt8\") pod \"payload-processing-external-model-5bcd5df458-7z9tf\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") " pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:22.883531 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.883490    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:22.921010 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:22.920948    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:23.292792 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:23.292763    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"]
Nov 04 03:38:23.299560 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:38:23.299526    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4465d326_a49f_4c83_8486_7d8efebcc905.slice/crio-53235d9e8741ad5862074dd656107e498dd8837fe87fed66ab0cabbc770b8ce0 WatchSource:0}: Error finding container 53235d9e8741ad5862074dd656107e498dd8837fe87fed66ab0cabbc770b8ce0: Status 404 returned error can't find the container with id 53235d9e8741ad5862074dd656107e498dd8837fe87fed66ab0cabbc770b8ce0
Nov 04 03:38:23.819969 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:23.819934    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf" event={"ID":"4465d326-a49f-4c83-8486-7d8efebcc905","Type":"ContainerStarted","Data":"51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef"}
Nov 04 03:38:23.820416 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:23.819989    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf" event={"ID":"4465d326-a49f-4c83-8486-7d8efebcc905","Type":"ContainerStarted","Data":"53235d9e8741ad5862074dd656107e498dd8837fe87fed66ab0cabbc770b8ce0"}
Nov 04 03:38:23.820416 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:23.820038    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:23.931765 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:23.931715    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf" podStartSLOduration=1.9317019279999998 podStartE2EDuration="1.931701928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:38:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:23.929480285 +0000 UTC m=+1273.744271563" watchObservedRunningTime="2026-11-04 03:38:23.931701928 +0000 UTC m=+1273.746493205"
Nov 04 03:38:26.213222 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:26.213180    4420 patch_prober.go:29] interesting pod/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:38:26.213618 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:26.213241    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" podUID="78034d31-2927-4a4d-aefe-85df17ec82c9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:38:26.862164 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:26.862126    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:27.066991 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.066954    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:27.213190 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.213169    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:38:27.265488 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.265466    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-kube-api-access-cjsjb\" (UniqueName: \"kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-kube-api-access-cjsjb\") pod \"78034d31-2927-4a4d-aefe-85df17ec82c9\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") "
Nov 04 03:38:27.265946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.265495    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78034d31-2927-4a4d-aefe-85df17ec82c9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78034d31-2927-4a4d-aefe-85df17ec82c9-istiod-ca-cert\") pod \"78034d31-2927-4a4d-aefe-85df17ec82c9\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") "
Nov 04 03:38:27.265946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.265527    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-data\") pod \"78034d31-2927-4a4d-aefe-85df17ec82c9\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") "
Nov 04 03:38:27.265946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.265553    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-socket\") pod \"78034d31-2927-4a4d-aefe-85df17ec82c9\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") "
Nov 04 03:38:27.265946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.265580    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-certs\") pod \"78034d31-2927-4a4d-aefe-85df17ec82c9\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") "
Nov 04 03:38:27.265946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.265606    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-envoy\") pod \"78034d31-2927-4a4d-aefe-85df17ec82c9\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") "
Nov 04 03:38:27.265946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.265684    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-podinfo\") pod \"78034d31-2927-4a4d-aefe-85df17ec82c9\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") "
Nov 04 03:38:27.265946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.265749    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-token\" (UniqueName: \"kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-token\") pod \"78034d31-2927-4a4d-aefe-85df17ec82c9\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") "
Nov 04 03:38:27.265946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.265875    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-socket" pod "78034d31-2927-4a4d-aefe-85df17ec82c9" (UID: "78034d31-2927-4a4d-aefe-85df17ec82c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:38:27.265946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.265927    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-data" pod "78034d31-2927-4a4d-aefe-85df17ec82c9" (UID: "78034d31-2927-4a4d-aefe-85df17ec82c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:38:27.265946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.265946    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-credential-socket\") pod \"78034d31-2927-4a4d-aefe-85df17ec82c9\" (UID: \"78034d31-2927-4a4d-aefe-85df17ec82c9\") "
Nov 04 03:38:27.266448 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.265953    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78034d31-2927-4a4d-aefe-85df17ec82c9-istiod-ca-cert" pod "78034d31-2927-4a4d-aefe-85df17ec82c9" (UID: "78034d31-2927-4a4d-aefe-85df17ec82c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:38:27.266448 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.266223    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-certs" pod "78034d31-2927-4a4d-aefe-85df17ec82c9" (UID: "78034d31-2927-4a4d-aefe-85df17ec82c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:38:27.266448 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.266233    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-credential-socket" pod "78034d31-2927-4a4d-aefe-85df17ec82c9" (UID: "78034d31-2927-4a4d-aefe-85df17ec82c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:38:27.266448 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.266382    4420 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-credential-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:27.266448 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.266404    4420 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78034d31-2927-4a4d-aefe-85df17ec82c9-istiod-ca-cert\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:27.266448 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.266419    4420 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-data\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:27.266448 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.266447    4420 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:27.266826 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.266462    4420 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-workload-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:27.267862 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.267819    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-kube-api-access-cjsjb" pod "78034d31-2927-4a4d-aefe-85df17ec82c9" (UID: "78034d31-2927-4a4d-aefe-85df17ec82c9"). InnerVolumeSpecName "kube-api-access-cjsjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:38:27.267964 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.267888    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-token" pod "78034d31-2927-4a4d-aefe-85df17ec82c9" (UID: "78034d31-2927-4a4d-aefe-85df17ec82c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:38:27.268084 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.268066    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-podinfo" pod "78034d31-2927-4a4d-aefe-85df17ec82c9" (UID: "78034d31-2927-4a4d-aefe-85df17ec82c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:38:27.268268 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.268253    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-envoy" pod "78034d31-2927-4a4d-aefe-85df17ec82c9" (UID: "78034d31-2927-4a4d-aefe-85df17ec82c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:38:27.367334 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.367301    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjsjb\" (UniqueName: \"kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-kube-api-access-cjsjb\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:27.367334 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.367335    4420 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-envoy\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:27.367538 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.367349    4420 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-podinfo\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:27.367538 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.367361    4420 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78034d31-2927-4a4d-aefe-85df17ec82c9-istio-token\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:27.754417 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.754380    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:27.842968 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.842933    4420 generic.go:356] "Generic (PLEG): container finished" podID="78034d31-2927-4a4d-aefe-85df17ec82c9" containerID="820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a" exitCode=0
Nov 04 03:38:27.843142 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.842974    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" event={"ID":"78034d31-2927-4a4d-aefe-85df17ec82c9","Type":"ContainerDied","Data":"820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a"}
Nov 04 03:38:27.843142 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.842979    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"
Nov 04 03:38:27.843142 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.843019    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj" event={"ID":"78034d31-2927-4a4d-aefe-85df17ec82c9","Type":"ContainerDied","Data":"a223de7f077704ce5c7580fbe812618d0ce50945efa3adf4e77bb708831cf8f4"}
Nov 04 03:38:27.843142 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.843042    4420 scope.go:122] "RemoveContainer" containerID="820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a"
Nov 04 03:38:27.853699 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.853682    4420 scope.go:122] "RemoveContainer" containerID="820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a"
Nov 04 03:38:27.854023 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:38:27.853988    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a\": container with ID starting with 820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a not found: ID does not exist" containerID="820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a"
Nov 04 03:38:27.854087 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.854033    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a"} err="failed to get container status \"820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a\": rpc error: code = NotFound desc = could not find container \"820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a\": container with ID starting with 820b4844bede439a1290fc9e486ac897e81f4f3c031dad97b3ba994ef29f5f7a not found: ID does not exist"
Nov 04 03:38:27.996868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:27.996826    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"]
Nov 04 03:38:28.003769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:28.003746    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-82c724-openshift-default-67ff8c8875-v42bj"]
Nov 04 03:38:28.307396 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:28.307364    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78034d31-2927-4a4d-aefe-85df17ec82c9" path="/var/lib/kubelet/pods/78034d31-2927-4a4d-aefe-85df17ec82c9/volumes"
Nov 04 03:38:30.832466 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:30.832435    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"]
Nov 04 03:38:30.832838 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:30.832824    4420 cpu_manager.go:414] "containerMap: removing container" podUID="78034d31-2927-4a4d-aefe-85df17ec82c9" containerName="istio-proxy"
Nov 04 03:38:30.832838 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:30.832838    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78034d31-2927-4a4d-aefe-85df17ec82c9" containerName="istio-proxy"
Nov 04 03:38:30.832948 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:30.832940    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="78034d31-2927-4a4d-aefe-85df17ec82c9" containerName="istio-proxy"
Nov 04 03:38:30.836343 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:30.836324    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:30.885435 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:30.885408    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-8b97550813bf\""
Nov 04 03:38:30.902426 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:30.902397    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials\") pod \"payload-processing-external-model-7f5c68695b-7z22d\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") " pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:30.902564 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:30.902435    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-plugins-config-volume\") pod \"payload-processing-external-model-7f5c68695b-7z22d\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") " pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:30.902564 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:30.902466    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtkqd\" (UniqueName: \"kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-kube-api-access-xtkqd\") pod \"payload-processing-external-model-7f5c68695b-7z22d\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") " pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:30.902691 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:30.902600    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-routing-overlay-volume\") pod \"payload-processing-external-model-7f5c68695b-7z22d\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") " pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:30.934060 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:30.934034    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"]
Nov 04 03:38:31.003665 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.003640    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials\") pod \"payload-processing-external-model-7f5c68695b-7z22d\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") " pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:31.003793 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.003671    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-plugins-config-volume\") pod \"payload-processing-external-model-7f5c68695b-7z22d\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") " pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:31.003793 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.003697    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtkqd\" (UniqueName: \"kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-kube-api-access-xtkqd\") pod \"payload-processing-external-model-7f5c68695b-7z22d\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") " pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:31.003793 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.003742    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-routing-overlay-volume\") pod \"payload-processing-external-model-7f5c68695b-7z22d\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") " pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:31.004397 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.004379    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-routing-overlay-volume\") pod \"payload-processing-external-model-7f5c68695b-7z22d\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") " pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:31.004454 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.004396    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-plugins-config-volume\") pod \"payload-processing-external-model-7f5c68695b-7z22d\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") " pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:31.006165 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.006143    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials\") pod \"payload-processing-external-model-7f5c68695b-7z22d\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") " pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:31.061837 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.061812    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtkqd\" (UniqueName: \"kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-kube-api-access-xtkqd\") pod \"payload-processing-external-model-7f5c68695b-7z22d\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") " pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:31.149664 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.149603    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:31.532688 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.532649    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"]
Nov 04 03:38:31.539586 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:38:31.539556    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8165ded7_cbdd_41e6_9790_06ad19ab9a2e.slice/crio-4322c689b52e4e4a4c5a95a085c04739a489d2c666e1b7aad98be3ea50cafdb9 WatchSource:0}: Error finding container 4322c689b52e4e4a4c5a95a085c04739a489d2c666e1b7aad98be3ea50cafdb9: Status 404 returned error can't find the container with id 4322c689b52e4e4a4c5a95a085c04739a489d2c666e1b7aad98be3ea50cafdb9
Nov 04 03:38:31.864895 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.864814    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d" event={"ID":"8165ded7-cbdd-41e6-9790-06ad19ab9a2e","Type":"ContainerStarted","Data":"fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea"}
Nov 04 03:38:31.864895 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.864888    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d" event={"ID":"8165ded7-cbdd-41e6-9790-06ad19ab9a2e","Type":"ContainerStarted","Data":"4322c689b52e4e4a4c5a95a085c04739a489d2c666e1b7aad98be3ea50cafdb9"}
Nov 04 03:38:31.865297 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.864939    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:31.975308 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:31.975259    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d" podStartSLOduration=1.9752461829999999 podStartE2EDuration="1.975246183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:38:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:31.974630797 +0000 UTC m=+1281.789422074" watchObservedRunningTime="2026-11-04 03:38:31.975246183 +0000 UTC m=+1281.790037460"
Nov 04 03:38:32.649438 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:38:32.649397    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:38:32.830851 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:32.830814    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:34.827793 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:34.827764    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:38.052257 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:38.052217    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:38.249223 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:38.249179    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:38.451422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:38.451350    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:42.873431 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:42.873405    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:38:43.055405 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:43.055364    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"]
Nov 04 03:38:43.251571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:43.251534    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:48.056694 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.056652    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf" podUID="4465d326-a49f-4c83-8486-7d8efebcc905" containerName="payload-processing" containerID="cri-o://51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef" gracePeriod=25
Nov 04 03:38:48.324258 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.324196    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:48.380606 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.380579    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-plugins-config-volume\") pod \"4465d326-a49f-4c83-8486-7d8efebcc905\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") "
Nov 04 03:38:48.380721 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.380641    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-provider-credentials\" (UniqueName: \"kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-provider-credentials\") pod \"4465d326-a49f-4c83-8486-7d8efebcc905\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") "
Nov 04 03:38:48.380721 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.380688    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-routing-overlay-volume\") pod \"4465d326-a49f-4c83-8486-7d8efebcc905\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") "
Nov 04 03:38:48.380809 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.380716    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-kube-api-access-qkqt8\" (UniqueName: \"kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-kube-api-access-qkqt8\") pod \"4465d326-a49f-4c83-8486-7d8efebcc905\" (UID: \"4465d326-a49f-4c83-8486-7d8efebcc905\") "
Nov 04 03:38:48.381006 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.380982    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-plugins-config-volume" pod "4465d326-a49f-4c83-8486-7d8efebcc905" (UID: "4465d326-a49f-4c83-8486-7d8efebcc905"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:38:48.381094 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.381056    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-routing-overlay-volume" pod "4465d326-a49f-4c83-8486-7d8efebcc905" (UID: "4465d326-a49f-4c83-8486-7d8efebcc905"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:38:48.381312 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.381290    4420 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-routing-overlay-volume\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:48.381312 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.381312    4420 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4465d326-a49f-4c83-8486-7d8efebcc905-plugins-config-volume\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:48.382769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.382744    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-provider-credentials" pod "4465d326-a49f-4c83-8486-7d8efebcc905" (UID: "4465d326-a49f-4c83-8486-7d8efebcc905"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:38:48.382878 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.382780    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-kube-api-access-qkqt8" pod "4465d326-a49f-4c83-8486-7d8efebcc905" (UID: "4465d326-a49f-4c83-8486-7d8efebcc905"). InnerVolumeSpecName "kube-api-access-qkqt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:38:48.442054 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.442008    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:48.482542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.482514    4420 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-provider-credentials\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:48.482542 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.482538    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkqt8\" (UniqueName: \"kubernetes.io/projected/4465d326-a49f-4c83-8486-7d8efebcc905-kube-api-access-qkqt8\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:38:48.949243 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.949208    4420 generic.go:356] "Generic (PLEG): container finished" podID="4465d326-a49f-4c83-8486-7d8efebcc905" containerID="51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef" exitCode=0
Nov 04 03:38:48.949451 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.949255    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"
Nov 04 03:38:48.949451 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.949293    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf" event={"ID":"4465d326-a49f-4c83-8486-7d8efebcc905","Type":"ContainerDied","Data":"51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef"}
Nov 04 03:38:48.949451 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.949356    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf" event={"ID":"4465d326-a49f-4c83-8486-7d8efebcc905","Type":"ContainerDied","Data":"53235d9e8741ad5862074dd656107e498dd8837fe87fed66ab0cabbc770b8ce0"}
Nov 04 03:38:48.949451 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.949386    4420 scope.go:122] "RemoveContainer" containerID="51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef"
Nov 04 03:38:48.961525 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.961499    4420 scope.go:122] "RemoveContainer" containerID="51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef"
Nov 04 03:38:48.961817 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:38:48.961797    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef\": container with ID starting with 51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef not found: ID does not exist" containerID="51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef"
Nov 04 03:38:48.961901 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:48.961827    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef"} err="failed to get container status \"51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef\": rpc error: code = NotFound desc = could not find container \"51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef\": container with ID starting with 51bc09e6a03547da64a30b66f9c175ec2e376c416d480c3af3f143280e91c8ef not found: ID does not exist"
Nov 04 03:38:49.103452 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:49.103426    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"]
Nov 04 03:38:49.150054 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:49.150028    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-5bcd5df458-7z9tf"]
Nov 04 03:38:50.307434 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:50.307401    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4465d326-a49f-4c83-8486-7d8efebcc905" path="/var/lib/kubelet/pods/4465d326-a49f-4c83-8486-7d8efebcc905/volumes"
Nov 04 03:38:52.126849 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.126761    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"]
Nov 04 03:38:52.127206 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.127190    4420 cpu_manager.go:414] "containerMap: removing container" podUID="4465d326-a49f-4c83-8486-7d8efebcc905" containerName="payload-processing"
Nov 04 03:38:52.127206 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.127203    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4465d326-a49f-4c83-8486-7d8efebcc905" containerName="payload-processing"
Nov 04 03:38:52.127296 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.127287    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="4465d326-a49f-4c83-8486-7d8efebcc905" containerName="payload-processing"
Nov 04 03:38:52.131770 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.131748    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.173696 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.173672    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"]
Nov 04 03:38:52.177210 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.177196    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.181002 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.180981    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dk4hf\""
Nov 04 03:38:52.218997 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.218969    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/094e67aa-800c-4f21-826f-2240183d969e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.219095 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219011    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.219095 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219038    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/094e67aa-800c-4f21-826f-2240183d969e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.219169 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219136    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.219255 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219186    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.219295 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219243    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.219295 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219276    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.219396 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219293    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.219396 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219355    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.219466 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219407    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.219466 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219438    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.219546 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219508    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.219546 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219541    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.219633 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219562    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/094e67aa-800c-4f21-826f-2240183d969e-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.219633 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219580    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.219704 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219661    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.219735 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219700    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng7zt\" (UniqueName: \"kubernetes.io/projected/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-kube-api-access-ng7zt\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.219735 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.219723    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch868\" (UniqueName: \"kubernetes.io/projected/094e67aa-800c-4f21-826f-2240183d969e-kube-api-access-ch868\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.226559 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.226538    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"]
Nov 04 03:38:52.237561 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.237536    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"]
Nov 04 03:38:52.321074 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321043    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.321236 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321086    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/094e67aa-800c-4f21-826f-2240183d969e-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.321236 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321212    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.321339 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321295    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.321393 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321369    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng7zt\" (UniqueName: \"kubernetes.io/projected/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-kube-api-access-ng7zt\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.321457 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321407    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch868\" (UniqueName: \"kubernetes.io/projected/094e67aa-800c-4f21-826f-2240183d969e-kube-api-access-ch868\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.321515 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321463    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/094e67aa-800c-4f21-826f-2240183d969e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.321515 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321496    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.321644 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321521    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/094e67aa-800c-4f21-826f-2240183d969e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.321644 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321612    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.321758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321690    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.321758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321714    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.321758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321747    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.321970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321790    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.321970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321833    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.321970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321913    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.321970 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321942    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.322172 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.321995    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.322172 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.322021    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.322172 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.322028    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.322172 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.322047    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.322386 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.322307    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.322386 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.322340    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/094e67aa-800c-4f21-826f-2240183d969e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.322386 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.322364    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.322513 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.322437    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.322513 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.322453    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.322623 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.322583    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.322623 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.322608    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.324371 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.324350    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/094e67aa-800c-4f21-826f-2240183d969e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.324482 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.324399    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.324482 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.324423    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/094e67aa-800c-4f21-826f-2240183d969e-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.324554 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.324517    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.378708 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.378652    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.379098 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.379076    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/094e67aa-800c-4f21-826f-2240183d969e-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.379226 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.379212    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch868\" (UniqueName: \"kubernetes.io/projected/094e67aa-800c-4f21-826f-2240183d969e-kube-api-access-ch868\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-j6k8d\" (UID: \"094e67aa-800c-4f21-826f-2240183d969e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.379315 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.379294    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng7zt\" (UniqueName: \"kubernetes.io/projected/dae825f6-0aa1-45e3-bba4-c2c83d6d9366-kube-api-access-ng7zt\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd2nl\" (UID: \"dae825f6-0aa1-45e3-bba4-c2c83d6d9366\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.444333 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.444312    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:52.489321 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.489294    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:52.872067 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.872034    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"]
Nov 04 03:38:52.875339 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.875317    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"]
Nov 04 03:38:52.878287 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:38:52.878261    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddae825f6_0aa1_45e3_bba4_c2c83d6d9366.slice/crio-c55ea3f3b50c8fcf293f120cd1dff0f5b3103170d2f457999c52c9c2cfb282f2 WatchSource:0}: Error finding container c55ea3f3b50c8fcf293f120cd1dff0f5b3103170d2f457999c52c9c2cfb282f2: Status 404 returned error can't find the container with id c55ea3f3b50c8fcf293f120cd1dff0f5b3103170d2f457999c52c9c2cfb282f2
Nov 04 03:38:52.880750 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.880713    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:38:52.880838 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.880790    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:38:52.880920 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.880833    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:38:52.880920 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:38:52.880896    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod094e67aa_800c_4f21_826f_2240183d969e.slice/crio-5dbd8a7db76c649c81623a301c4f3ea0ff9c4ed61b88583812b668861ecfb8bc WatchSource:0}: Error finding container 5dbd8a7db76c649c81623a301c4f3ea0ff9c4ed61b88583812b668861ecfb8bc: Status 404 returned error can't find the container with id 5dbd8a7db76c649c81623a301c4f3ea0ff9c4ed61b88583812b668861ecfb8bc
Nov 04 03:38:52.883121 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.883097    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:38:52.883198 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.883160    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:38:52.883198 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.883190    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:38:52.973013 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.972264    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d" event={"ID":"094e67aa-800c-4f21-826f-2240183d969e","Type":"ContainerStarted","Data":"0ad1b1c89d2df383f9b71bf7b5f165a2a2b6efdcf60ed6ce45c71852b14aa241"}
Nov 04 03:38:52.973013 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.972325    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d" event={"ID":"094e67aa-800c-4f21-826f-2240183d969e","Type":"ContainerStarted","Data":"5dbd8a7db76c649c81623a301c4f3ea0ff9c4ed61b88583812b668861ecfb8bc"}
Nov 04 03:38:52.975441 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.975227    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl" event={"ID":"dae825f6-0aa1-45e3-bba4-c2c83d6d9366","Type":"ContainerStarted","Data":"423707d2746123f8796c29ada281e32814eed6751b0feb2bc2fce5a5ed1761cb"}
Nov 04 03:38:52.975441 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:52.975269    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl" event={"ID":"dae825f6-0aa1-45e3-bba4-c2c83d6d9366","Type":"ContainerStarted","Data":"c55ea3f3b50c8fcf293f120cd1dff0f5b3103170d2f457999c52c9c2cfb282f2"}
Nov 04 03:38:53.088227 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:53.088168    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d" podStartSLOduration=1.088151777 podStartE2EDuration="1.088151777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:38:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:53.087179518 +0000 UTC m=+1302.901970796" watchObservedRunningTime="2026-11-04 03:38:53.088151777 +0000 UTC m=+1302.902943061"
Nov 04 03:38:53.201913 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:53.201095    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl" podStartSLOduration=1.201079094 podStartE2EDuration="1.201079094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:38:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:38:53.200614138 +0000 UTC m=+1303.015405407" watchObservedRunningTime="2026-11-04 03:38:53.201079094 +0000 UTC m=+1303.015870372"
Nov 04 03:38:53.315401 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:53.315369    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:53.444592 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:53.444563    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:38:53.446062 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:53.446034    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 04 03:38:53.446202 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:53.446086    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl" podUID="dae825f6-0aa1-45e3-bba4-c2c83d6d9366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 04 03:38:53.489631 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:53.489604    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:38:53.491143 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:53.491119    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 04 03:38:53.491259 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:53.491167    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d" podUID="094e67aa-800c-4f21-826f-2240183d969e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 04 03:38:54.463288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:54.463233    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 04 03:38:54.463885 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:54.463833    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl" podUID="dae825f6-0aa1-45e3-bba4-c2c83d6d9366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 04 03:38:54.498864 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:54.494302    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 04 03:38:54.498864 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:54.494373    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d" podUID="094e67aa-800c-4f21-826f-2240183d969e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 04 03:38:55.448868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:55.445218    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 04 03:38:55.448868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:55.445293    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl" podUID="dae825f6-0aa1-45e3-bba4-c2c83d6d9366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 04 03:38:55.493869 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:55.490140    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 04 03:38:55.493869 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:55.490198    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d" podUID="094e67aa-800c-4f21-826f-2240183d969e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 04 03:38:55.966867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:55.966701    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:38:56.448868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:56.448193    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 04 03:38:56.448868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:56.448265    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl" podUID="dae825f6-0aa1-45e3-bba4-c2c83d6d9366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 04 03:38:56.490871 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:56.490149    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 04 03:38:56.490871 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:56.490208    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d" podUID="094e67aa-800c-4f21-826f-2240183d969e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 04 03:38:57.448868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:57.448195    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 04 03:38:57.448868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:57.448257    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl" podUID="dae825f6-0aa1-45e3-bba4-c2c83d6d9366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 04 03:38:57.491866 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:57.490138    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 04 03:38:57.491866 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:57.490208    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d" podUID="094e67aa-800c-4f21-826f-2240183d969e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 04 03:38:58.325977 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:58.325933    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:38:58.449877 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:58.448142    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 04 03:38:58.449877 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:58.448200    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl" podUID="dae825f6-0aa1-45e3-bba4-c2c83d6d9366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 04 03:38:58.493867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:58.493208    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 04 03:38:58.493867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:58.493269    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d" podUID="094e67aa-800c-4f21-826f-2240183d969e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 04 03:38:59.444987 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:59.444948    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 04 03:38:59.445163 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:59.445015    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl" podUID="dae825f6-0aa1-45e3-bba4-c2c83d6d9366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 04 03:38:59.490798 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:59.490769    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 04 03:38:59.491159 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:38:59.490816    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d" podUID="094e67aa-800c-4f21-826f-2240183d969e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 04 03:39:00.445005 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:00.444964    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 04 03:39:00.445179 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:00.445031    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl" podUID="dae825f6-0aa1-45e3-bba4-c2c83d6d9366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 04 03:39:00.490708 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:00.490671    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 04 03:39:00.490886 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:00.490746    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d" podUID="094e67aa-800c-4f21-826f-2240183d969e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 04 03:39:01.885910 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:01.885876    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:39:01.887149 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:01.887123    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:39:01.887856 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:01.887819    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-j6k8d"
Nov 04 03:39:02.088851 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:02.088812    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:39:02.089350 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:02.089285    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:39:02.090433 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:02.090414    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd2nl"
Nov 04 03:39:02.650311 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:02.650269    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:39:03.268510 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:03.268427    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:06.637087 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:06.637053    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:07.233232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.233179    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:07.275350 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.275319    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"]
Nov 04 03:39:07.279046 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.279025    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.379298 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.379267    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"]
Nov 04 03:39:07.385313 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.385270    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:07.450666 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.450637    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.450821 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.450693    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.450892 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.450864    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.450941 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.450922    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e18db8b-6a81-49e5-9524-1b65624cac59-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.450984 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.450968    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.451019 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.450996    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.451049 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.451026    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjm8m\" (UniqueName: \"kubernetes.io/projected/3e18db8b-6a81-49e5-9524-1b65624cac59-kube-api-access-zjm8m\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.451082 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.451048    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e18db8b-6a81-49e5-9524-1b65624cac59-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.451118 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.451099    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e18db8b-6a81-49e5-9524-1b65624cac59-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.552351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.552321    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.552537 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.552371    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.552537 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.552417    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.552537 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.552445    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e18db8b-6a81-49e5-9524-1b65624cac59-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.552537 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.552481    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.552537 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.552508    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.552807 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.552551    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjm8m\" (UniqueName: \"kubernetes.io/projected/3e18db8b-6a81-49e5-9524-1b65624cac59-kube-api-access-zjm8m\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.552807 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.552598    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e18db8b-6a81-49e5-9524-1b65624cac59-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.552807 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.552628    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e18db8b-6a81-49e5-9524-1b65624cac59-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.553068 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.552811    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.553068 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.552875    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.553198 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.553104    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.553288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.553267    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e18db8b-6a81-49e5-9524-1b65624cac59-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.553376 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.553359    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.555157 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.555132    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e18db8b-6a81-49e5-9524-1b65624cac59-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.555251 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.555207    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e18db8b-6a81-49e5-9524-1b65624cac59-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.609738 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.609711    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e18db8b-6a81-49e5-9524-1b65624cac59-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.610004 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.609987    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjm8m\" (UniqueName: \"kubernetes.io/projected/3e18db8b-6a81-49e5-9524-1b65624cac59-kube-api-access-zjm8m\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-jx8n4\" (UID: \"3e18db8b-6a81-49e5-9524-1b65624cac59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:07.892524 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:07.892451    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:08.218818 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.218740    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"]
Nov 04 03:39:08.222914 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.222887    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:08.274627 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.274597    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 03:39:08.321082 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.321052    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:39:08.321248 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.321232    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 03:39:08.321306 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.321267    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:39:08.323410 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.323391    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"]
Nov 04 03:39:08.361639 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.361610    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f6219028-c4clw\" (UID: \"2628336f-46ef-49fe-bc60-0d0a4db688cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:08.361789 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.361656    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f6219028-c4clw\" (UID: \"2628336f-46ef-49fe-bc60-0d0a4db688cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:08.361880 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.361833    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wsbv\" (UniqueName: \"kubernetes.io/projected/2628336f-46ef-49fe-bc60-0d0a4db688cf-kube-api-access-4wsbv\") pod \"maas-api-revoke-sub-f6219028-c4clw\" (UID: \"2628336f-46ef-49fe-bc60-0d0a4db688cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:08.377008 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.376980    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"]
Nov 04 03:39:08.392643 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:39:08.392617    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e18db8b_6a81_49e5_9524_1b65624cac59.slice/crio-cecac9ffa52d7e9f70b061ab47da905e50a1e89f569075eeef7ce49e1edf958c WatchSource:0}: Error finding container cecac9ffa52d7e9f70b061ab47da905e50a1e89f569075eeef7ce49e1edf958c: Status 404 returned error can't find the container with id cecac9ffa52d7e9f70b061ab47da905e50a1e89f569075eeef7ce49e1edf958c
Nov 04 03:39:08.394960 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.394931    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:39:08.395019 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.395002    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:39:08.395055 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.395035    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:39:08.436753 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.436727    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:08.463127 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.463103    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsbv\" (UniqueName: \"kubernetes.io/projected/2628336f-46ef-49fe-bc60-0d0a4db688cf-kube-api-access-4wsbv\") pod \"maas-api-revoke-sub-f6219028-c4clw\" (UID: \"2628336f-46ef-49fe-bc60-0d0a4db688cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:08.463240 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.463147    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f6219028-c4clw\" (UID: \"2628336f-46ef-49fe-bc60-0d0a4db688cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:08.463240 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.463198    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f6219028-c4clw\" (UID: \"2628336f-46ef-49fe-bc60-0d0a4db688cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:08.463958 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.463941    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f6219028-c4clw\" (UID: \"2628336f-46ef-49fe-bc60-0d0a4db688cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:08.465273 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.465258    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f6219028-c4clw\" (UID: \"2628336f-46ef-49fe-bc60-0d0a4db688cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:08.520896 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.520859    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wsbv\" (UniqueName: \"kubernetes.io/projected/2628336f-46ef-49fe-bc60-0d0a4db688cf-kube-api-access-4wsbv\") pod \"maas-api-revoke-sub-f6219028-c4clw\" (UID: \"2628336f-46ef-49fe-bc60-0d0a4db688cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:08.536226 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.536186    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:08.912497 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:08.912420    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"]
Nov 04 03:39:08.919040 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:39:08.919010    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2628336f_46ef_49fe_bc60_0d0a4db688cf.slice/crio-676a2e82ecc2ee8d6b45f0ea448b33b64d546996cc325d178449ae366b807172 WatchSource:0}: Error finding container 676a2e82ecc2ee8d6b45f0ea448b33b64d546996cc325d178449ae366b807172: Status 404 returned error can't find the container with id 676a2e82ecc2ee8d6b45f0ea448b33b64d546996cc325d178449ae366b807172
Nov 04 03:39:09.082396 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:09.082345    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw" event={"ID":"2628336f-46ef-49fe-bc60-0d0a4db688cf","Type":"ContainerStarted","Data":"676a2e82ecc2ee8d6b45f0ea448b33b64d546996cc325d178449ae366b807172"}
Nov 04 03:39:09.083817 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:09.083781    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4" event={"ID":"3e18db8b-6a81-49e5-9524-1b65624cac59","Type":"ContainerStarted","Data":"85e242fb568e848fa4309c1290be3523f648768e9025d4842a2ebb344d65ea70"}
Nov 04 03:39:09.083940 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:09.083832    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4" event={"ID":"3e18db8b-6a81-49e5-9524-1b65624cac59","Type":"ContainerStarted","Data":"cecac9ffa52d7e9f70b061ab47da905e50a1e89f569075eeef7ce49e1edf958c"}
Nov 04 03:39:09.198296 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:09.198193    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4" podStartSLOduration=2.198179833 podStartE2EDuration="2.198179833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:39:09.196811764 +0000 UTC m=+1319.011603041" watchObservedRunningTime="2026-11-04 03:39:09.198179833 +0000 UTC m=+1319.012971184"
Nov 04 03:39:09.893109 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:09.893065    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:09.894806 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:09.894778    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 04 03:39:09.894954 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:09.894833    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4" podUID="3e18db8b-6a81-49e5-9524-1b65624cac59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 04 03:39:10.892793 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:10.892760    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 04 03:39:10.893257 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:10.892810    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4" podUID="3e18db8b-6a81-49e5-9524-1b65624cac59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 04 03:39:11.893576 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:11.893530    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 04 03:39:11.893973 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:11.893595    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4" podUID="3e18db8b-6a81-49e5-9524-1b65624cac59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 04 03:39:12.893358 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:12.893322    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 04 03:39:12.893524 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:12.893378    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4" podUID="3e18db8b-6a81-49e5-9524-1b65624cac59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 04 03:39:13.893013 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:13.892987    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 04 03:39:13.893366 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:13.893031    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4" podUID="3e18db8b-6a81-49e5-9524-1b65624cac59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 04 03:39:13.968523 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:13.968443    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:14.893538 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:14.893496    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 04 03:39:14.893951 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:14.893554    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4" podUID="3e18db8b-6a81-49e5-9524-1b65624cac59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 04 03:39:15.120804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:15.120771    4420 generic.go:356] "Generic (PLEG): container finished" podID="2628336f-46ef-49fe-bc60-0d0a4db688cf" containerID="cc30605ba16218069f0838db5a407b55838d1a85ddaa65e6e5b38fef2b9c6d54" exitCode=0
Nov 04 03:39:15.121017 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:15.120876    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw" event={"ID":"2628336f-46ef-49fe-bc60-0d0a4db688cf","Type":"ContainerDied","Data":"cc30605ba16218069f0838db5a407b55838d1a85ddaa65e6e5b38fef2b9c6d54"}
Nov 04 03:39:15.892727 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:15.892695    4420 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 04 03:39:15.893014 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:15.892740    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4" podUID="3e18db8b-6a81-49e5-9524-1b65624cac59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 04 03:39:16.277638 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:16.277608    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:16.444787 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:16.444702    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2628336f-46ef-49fe-bc60-0d0a4db688cf-kube-api-access-4wsbv\" (UniqueName: \"kubernetes.io/projected/2628336f-46ef-49fe-bc60-0d0a4db688cf-kube-api-access-4wsbv\") pod \"2628336f-46ef-49fe-bc60-0d0a4db688cf\" (UID: \"2628336f-46ef-49fe-bc60-0d0a4db688cf\") "
Nov 04 03:39:16.444992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:16.444822    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-openshift-ca\") pod \"2628336f-46ef-49fe-bc60-0d0a4db688cf\" (UID: \"2628336f-46ef-49fe-bc60-0d0a4db688cf\") "
Nov 04 03:39:16.444992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:16.444881    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-xks-ca\") pod \"2628336f-46ef-49fe-bc60-0d0a4db688cf\" (UID: \"2628336f-46ef-49fe-bc60-0d0a4db688cf\") "
Nov 04 03:39:16.445266 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:16.445233    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-openshift-ca" pod "2628336f-46ef-49fe-bc60-0d0a4db688cf" (UID: "2628336f-46ef-49fe-bc60-0d0a4db688cf"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:39:16.447172 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:16.447142    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-xks-ca" pod "2628336f-46ef-49fe-bc60-0d0a4db688cf" (UID: "2628336f-46ef-49fe-bc60-0d0a4db688cf"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:39:16.447266 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:16.447173    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2628336f-46ef-49fe-bc60-0d0a4db688cf-kube-api-access-4wsbv" pod "2628336f-46ef-49fe-bc60-0d0a4db688cf" (UID: "2628336f-46ef-49fe-bc60-0d0a4db688cf"). InnerVolumeSpecName "kube-api-access-4wsbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:39:16.546773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:16.546740    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-openshift-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:16.546903 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:16.546775    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2628336f-46ef-49fe-bc60-0d0a4db688cf-maas-api-xks-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:16.546903 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:16.546793    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wsbv\" (UniqueName: \"kubernetes.io/projected/2628336f-46ef-49fe-bc60-0d0a4db688cf-kube-api-access-4wsbv\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:17.133602 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:17.133560    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw" event={"ID":"2628336f-46ef-49fe-bc60-0d0a4db688cf","Type":"ContainerDied","Data":"676a2e82ecc2ee8d6b45f0ea448b33b64d546996cc325d178449ae366b807172"}
Nov 04 03:39:17.133794 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:17.133624    4420 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="676a2e82ecc2ee8d6b45f0ea448b33b64d546996cc325d178449ae366b807172"
Nov 04 03:39:17.133794 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:17.133582    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"
Nov 04 03:39:17.274387 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:17.274355    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:17.274894 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:17.274877    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:17.275665 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:17.275645    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-jx8n4"
Nov 04 03:39:18.542685 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:18.542653    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:19.105400 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:19.105368    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:19.394602 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:19.394526    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:20.904522 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:20.904484    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:21.056022 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:21.055986    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:21.211173 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:21.211099    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:21.580293 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:21.580247    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:21.735318 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:21.735268    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:23.267016 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:23.266971    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:23.414516 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:23.414482    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:23.574470 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:23.574385    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:24.608211 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:24.608173    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:25.221395 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.221365    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"]
Nov 04 03:39:25.222028 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.222007    4420 cpu_manager.go:414] "containerMap: removing container" podUID="2628336f-46ef-49fe-bc60-0d0a4db688cf" containerName="revoke-keys"
Nov 04 03:39:25.222028 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.222027    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2628336f-46ef-49fe-bc60-0d0a4db688cf" containerName="revoke-keys"
Nov 04 03:39:25.222193 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.222128    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="2628336f-46ef-49fe-bc60-0d0a4db688cf" containerName="revoke-keys"
Nov 04 03:39:25.226529 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.226510    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:25.325127 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.325099    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 03:39:25.325290 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.325141    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:39:25.325290 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.325217    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:39:25.325290 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.325237    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 03:39:25.327543 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.327525    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"]
Nov 04 03:39:25.350092 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.350065    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds68l\" (UniqueName: \"kubernetes.io/projected/40d929de-fa04-41b7-8514-893046288682-kube-api-access-ds68l\") pod \"maas-api-revoke-sub-f3194190-whck8\" (UID: \"40d929de-fa04-41b7-8514-893046288682\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:25.350227 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.350122    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/40d929de-fa04-41b7-8514-893046288682-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f3194190-whck8\" (UID: \"40d929de-fa04-41b7-8514-893046288682\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:25.350284 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.350238    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/40d929de-fa04-41b7-8514-893046288682-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f3194190-whck8\" (UID: \"40d929de-fa04-41b7-8514-893046288682\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:25.451804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.451773    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds68l\" (UniqueName: \"kubernetes.io/projected/40d929de-fa04-41b7-8514-893046288682-kube-api-access-ds68l\") pod \"maas-api-revoke-sub-f3194190-whck8\" (UID: \"40d929de-fa04-41b7-8514-893046288682\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:25.451943 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.451864    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/40d929de-fa04-41b7-8514-893046288682-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f3194190-whck8\" (UID: \"40d929de-fa04-41b7-8514-893046288682\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:25.451943 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.451916    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/40d929de-fa04-41b7-8514-893046288682-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f3194190-whck8\" (UID: \"40d929de-fa04-41b7-8514-893046288682\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:25.452661 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.452642    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/40d929de-fa04-41b7-8514-893046288682-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f3194190-whck8\" (UID: \"40d929de-fa04-41b7-8514-893046288682\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:25.454298 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.454281    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/40d929de-fa04-41b7-8514-893046288682-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f3194190-whck8\" (UID: \"40d929de-fa04-41b7-8514-893046288682\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:25.509723 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.509702    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds68l\" (UniqueName: \"kubernetes.io/projected/40d929de-fa04-41b7-8514-893046288682-kube-api-access-ds68l\") pod \"maas-api-revoke-sub-f3194190-whck8\" (UID: \"40d929de-fa04-41b7-8514-893046288682\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:25.539660 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.539636    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:25.594875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.594815    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:25.908862 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:25.908766    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"]
Nov 04 03:39:25.915469 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:39:25.915442    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40d929de_fa04_41b7_8514_893046288682.slice/crio-c9022fd9b7a44ad77534805433ad1e5db0dabda4bf8f9914d2981304228925bb WatchSource:0}: Error finding container c9022fd9b7a44ad77534805433ad1e5db0dabda4bf8f9914d2981304228925bb: Status 404 returned error can't find the container with id c9022fd9b7a44ad77534805433ad1e5db0dabda4bf8f9914d2981304228925bb
Nov 04 03:39:26.140788 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.140750    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"]
Nov 04 03:39:26.146462 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.146439    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.202909 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.196546    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-951893-openshift-default-dockercfg-gtjqk\""
Nov 04 03:39:26.202909 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.200634    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8" event={"ID":"40d929de-fa04-41b7-8514-893046288682","Type":"ContainerStarted","Data":"c9022fd9b7a44ad77534805433ad1e5db0dabda4bf8f9914d2981304228925bb"}
Nov 04 03:39:26.240617 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.240584    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"]
Nov 04 03:39:26.268875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.262551    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-socket\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.268875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.262679    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-certs\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.268875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.262714    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istiod-ca-cert\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.268875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.262742    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-token\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.268875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.262790    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-envoy\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.268875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.262834    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-credential-socket\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.268875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.262911    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6xpm\" (UniqueName: \"kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-kube-api-access-g6xpm\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.268875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.263033    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-data\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.268875 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.263130    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-podinfo\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.276430 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.276368    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:26.364255 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.364214    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-certs\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.364422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.364262    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istiod-ca-cert\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.364422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.364292    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-token\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.364422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.364336    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-envoy\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.364422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.364377    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-credential-socket\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.364422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.364416    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6xpm\" (UniqueName: \"kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-kube-api-access-g6xpm\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.366151 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.364443    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-data\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.366151 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.364515    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-podinfo\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.366151 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.364562    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-socket\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.366151 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.364821    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-certs\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.366151 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.365205    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istiod-ca-cert\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.366151 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.365581    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-credential-socket\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.366151 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.365653    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-socket\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.366767 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.366738    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-data\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.367329 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.367313    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-envoy\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.368639 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.368616    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-podinfo\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.422724 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.422654    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6xpm\" (UniqueName: \"kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-kube-api-access-g6xpm\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.422724 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.422708    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-token\") pod \"e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") " pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.510018 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.509983    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:26.531130 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.531092    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:26.946742 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:26.946714    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"]
Nov 04 03:39:27.018753 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:39:27.018721    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a9287c7_4a73_4ec5_9fa3_b22e78c8b25e.slice/crio-9b7d4da662870392d3b446d529883cd99e068be61d1c636b4950cc3f1a09e436 WatchSource:0}: Error finding container 9b7d4da662870392d3b446d529883cd99e068be61d1c636b4950cc3f1a09e436: Status 404 returned error can't find the container with id 9b7d4da662870392d3b446d529883cd99e068be61d1c636b4950cc3f1a09e436
Nov 04 03:39:27.021302 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:27.021265    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:39:27.021379 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:27.021340    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:39:27.021379 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:27.021370    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:39:27.208563 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:27.208469    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" event={"ID":"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e","Type":"ContainerStarted","Data":"b114c09d6992206d59be7de814dba1a468fb8634556a64da10f048e20d2e27e1"}
Nov 04 03:39:27.208563 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:27.208557    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" event={"ID":"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e","Type":"ContainerStarted","Data":"9b7d4da662870392d3b446d529883cd99e068be61d1c636b4950cc3f1a09e436"}
Nov 04 03:39:27.210012 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:27.209984    4420 generic.go:356] "Generic (PLEG): container finished" podID="40d929de-fa04-41b7-8514-893046288682" containerID="ba2d8384dfa0b16b6a4d98c02bb13e46e302b7d82b30093b1380c64277ba77a2" exitCode=0
Nov 04 03:39:27.210144 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:27.210056    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8" event={"ID":"40d929de-fa04-41b7-8514-893046288682","Type":"ContainerDied","Data":"ba2d8384dfa0b16b6a4d98c02bb13e46e302b7d82b30093b1380c64277ba77a2"}
Nov 04 03:39:27.323294 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:27.323245    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" podStartSLOduration=1.323231304 podStartE2EDuration="1.323231304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:39:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:39:27.321396231 +0000 UTC m=+1337.136187510" watchObservedRunningTime="2026-11-04 03:39:27.323231304 +0000 UTC m=+1337.138022579"
Nov 04 03:39:27.510790 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:27.510750    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:28.366228 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.366205    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:28.487146 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.487065    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40d929de-fa04-41b7-8514-893046288682-kube-api-access-ds68l\" (UniqueName: \"kubernetes.io/projected/40d929de-fa04-41b7-8514-893046288682-kube-api-access-ds68l\") pod \"40d929de-fa04-41b7-8514-893046288682\" (UID: \"40d929de-fa04-41b7-8514-893046288682\") "
Nov 04 03:39:28.487146 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.487112    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40d929de-fa04-41b7-8514-893046288682-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/40d929de-fa04-41b7-8514-893046288682-maas-api-xks-ca\") pod \"40d929de-fa04-41b7-8514-893046288682\" (UID: \"40d929de-fa04-41b7-8514-893046288682\") "
Nov 04 03:39:28.487382 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.487332    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40d929de-fa04-41b7-8514-893046288682-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/40d929de-fa04-41b7-8514-893046288682-maas-api-openshift-ca\") pod \"40d929de-fa04-41b7-8514-893046288682\" (UID: \"40d929de-fa04-41b7-8514-893046288682\") "
Nov 04 03:39:28.487890 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.487858    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40d929de-fa04-41b7-8514-893046288682-maas-api-openshift-ca" pod "40d929de-fa04-41b7-8514-893046288682" (UID: "40d929de-fa04-41b7-8514-893046288682"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:39:28.489772 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.489743    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40d929de-fa04-41b7-8514-893046288682-maas-api-xks-ca" pod "40d929de-fa04-41b7-8514-893046288682" (UID: "40d929de-fa04-41b7-8514-893046288682"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:39:28.490230 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.490202    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40d929de-fa04-41b7-8514-893046288682-kube-api-access-ds68l" pod "40d929de-fa04-41b7-8514-893046288682" (UID: "40d929de-fa04-41b7-8514-893046288682"). InnerVolumeSpecName "kube-api-access-ds68l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:39:28.516596 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.516568    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:28.555273 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.555249    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"]
Nov 04 03:39:28.588315 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.588289    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/40d929de-fa04-41b7-8514-893046288682-maas-api-openshift-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:28.588315 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.588315    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds68l\" (UniqueName: \"kubernetes.io/projected/40d929de-fa04-41b7-8514-893046288682-kube-api-access-ds68l\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:28.588473 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.588325    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/40d929de-fa04-41b7-8514-893046288682-maas-api-xks-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:28.641784 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:28.641750    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:29.223452 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:29.223417    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8" event={"ID":"40d929de-fa04-41b7-8514-893046288682","Type":"ContainerDied","Data":"c9022fd9b7a44ad77534805433ad1e5db0dabda4bf8f9914d2981304228925bb"}
Nov 04 03:39:29.223643 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:29.223492    4420 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9022fd9b7a44ad77534805433ad1e5db0dabda4bf8f9914d2981304228925bb"
Nov 04 03:39:29.223643 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:29.223595    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:29.223643 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:29.223606    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"
Nov 04 03:39:29.224437 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:29.224414    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:39:29.353748 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:29.353717    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:29.659777 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:29.659746    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:30.240194 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.240161    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:30.460298 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.460262    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:30.516675 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.516602    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"]
Nov 04 03:39:30.517147 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.517130    4420 cpu_manager.go:414] "containerMap: removing container" podUID="40d929de-fa04-41b7-8514-893046288682" containerName="revoke-keys"
Nov 04 03:39:30.517221 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.517149    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40d929de-fa04-41b7-8514-893046288682" containerName="revoke-keys"
Nov 04 03:39:30.517288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.517239    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="40d929de-fa04-41b7-8514-893046288682" containerName="revoke-keys"
Nov 04 03:39:30.520394 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.520374    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.611296 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.611266    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") " pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.611449 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.611305    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") " pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.611548 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.611520    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") " pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.611588 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.611563    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l7xv\" (UniqueName: \"kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-kube-api-access-2l7xv\") pod \"payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") " pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.615603 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.615577    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:30.622703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.622686    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-1bf4bf\"/\"kube-root-ca.crt\""
Nov 04 03:39:30.622859 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.622822    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-1bf4bf\"/\"payload-processing-external-model-plugins-e2e-extmod-b-1bf4bf\""
Nov 04 03:39:30.622949 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.622865    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-1bf4bf\"/\"agc-cred-aefdcebaf7e5\""
Nov 04 03:39:30.622949 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.622828    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-1bf4bf\"/\"routing-overlay\""
Nov 04 03:39:30.622949 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.622872    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-1bf4bf\"/\"payload-processing-external-model-e2e--16932683-dockercfg-chwc5\""
Nov 04 03:39:30.622949 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.622907    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-1bf4bf\"/\"openshift-service-ca.crt\""
Nov 04 03:39:30.625299 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.625279    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"]
Nov 04 03:39:30.713182 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.713158    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") " pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.713573 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.713187    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") " pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.713573 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.713274    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") " pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.713573 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.713297    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l7xv\" (UniqueName: \"kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-kube-api-access-2l7xv\") pod \"payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") " pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.713815 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.713795    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") " pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.713959 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.713941    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") " pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.715718 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.715697    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") " pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.775764 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.775710    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l7xv\" (UniqueName: \"kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-kube-api-access-2l7xv\") pod \"payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") " pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:30.833071 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:30.833045    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:31.203609 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:31.203533    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"]
Nov 04 03:39:31.209987 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:39:31.209960    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3d2aa1f_c0c2_49a7_8b25_202c43d02133.slice/crio-9e3fa6d7a69047696385da8b295a6877401ff59f941021094b779eb7e966e51b WatchSource:0}: Error finding container 9e3fa6d7a69047696385da8b295a6877401ff59f941021094b779eb7e966e51b: Status 404 returned error can't find the container with id 9e3fa6d7a69047696385da8b295a6877401ff59f941021094b779eb7e966e51b
Nov 04 03:39:31.235779 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:31.235753    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9" event={"ID":"b3d2aa1f-c0c2-49a7-8b25-202c43d02133","Type":"ContainerStarted","Data":"9e3fa6d7a69047696385da8b295a6877401ff59f941021094b779eb7e966e51b"}
Nov 04 03:39:31.266293 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:31.266272    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:31.864869 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:31.864825    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"]
Nov 04 03:39:31.868858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:31.868824    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:31.926273 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:31.926240    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-provider-credentials\") pod \"payload-processing-external-model-8b89f9dc6-zbkqz\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") " pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:31.926403 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:31.926329    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-plugins-config-volume\") pod \"payload-processing-external-model-8b89f9dc6-zbkqz\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") " pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:31.926403 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:31.926385    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-routing-overlay-volume\") pod \"payload-processing-external-model-8b89f9dc6-zbkqz\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") " pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:31.926522 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:31.926418    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnthb\" (UniqueName: \"kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-kube-api-access-bnthb\") pod \"payload-processing-external-model-8b89f9dc6-zbkqz\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") " pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:31.926624 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:31.926604    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"]
Nov 04 03:39:32.027487 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.027445    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-routing-overlay-volume\") pod \"payload-processing-external-model-8b89f9dc6-zbkqz\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") " pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:32.027487 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.027484    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnthb\" (UniqueName: \"kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-kube-api-access-bnthb\") pod \"payload-processing-external-model-8b89f9dc6-zbkqz\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") " pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:32.027710 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.027651    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-provider-credentials\") pod \"payload-processing-external-model-8b89f9dc6-zbkqz\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") " pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:32.027768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.027718    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-plugins-config-volume\") pod \"payload-processing-external-model-8b89f9dc6-zbkqz\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") " pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:32.028400 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.028370    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-routing-overlay-volume\") pod \"payload-processing-external-model-8b89f9dc6-zbkqz\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") " pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:32.028574 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.028554    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-plugins-config-volume\") pod \"payload-processing-external-model-8b89f9dc6-zbkqz\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") " pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:32.030208 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.030185    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-provider-credentials\") pod \"payload-processing-external-model-8b89f9dc6-zbkqz\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") " pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:32.085423 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.085397    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnthb\" (UniqueName: \"kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-kube-api-access-bnthb\") pod \"payload-processing-external-model-8b89f9dc6-zbkqz\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") " pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:32.182142 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.182084    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:32.248612 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.248578    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9" event={"ID":"b3d2aa1f-c0c2-49a7-8b25-202c43d02133","Type":"ContainerStarted","Data":"b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3"}
Nov 04 03:39:32.248726 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.248713    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:32.359312 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.359262    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9" podStartSLOduration=2.359248521 podStartE2EDuration="2.359248521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:39:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:39:32.357916619 +0000 UTC m=+1342.172707897" watchObservedRunningTime="2026-11-04 03:39:32.359248521 +0000 UTC m=+1342.174039798"
Nov 04 03:39:32.421073 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.421034    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:32.556916 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:32.556881    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"]
Nov 04 03:39:32.562892 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:39:32.562823    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod156e2961_9ef5_43b5_8378_893caecef3d4.slice/crio-baba4e0f82f57d0e77b942be4c4ad2a4cc52345abf3fb873f4b49b2f65ee9d67 WatchSource:0}: Error finding container baba4e0f82f57d0e77b942be4c4ad2a4cc52345abf3fb873f4b49b2f65ee9d67: Status 404 returned error can't find the container with id baba4e0f82f57d0e77b942be4c4ad2a4cc52345abf3fb873f4b49b2f65ee9d67
Nov 04 03:39:32.649821 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:32.649785    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:39:33.255001 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:33.254956    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" event={"ID":"156e2961-9ef5-43b5-8378-893caecef3d4","Type":"ContainerStarted","Data":"a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a"}
Nov 04 03:39:33.255478 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:33.255027    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" event={"ID":"156e2961-9ef5-43b5-8378-893caecef3d4","Type":"ContainerStarted","Data":"baba4e0f82f57d0e77b942be4c4ad2a4cc52345abf3fb873f4b49b2f65ee9d67"}
Nov 04 03:39:33.255478 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:33.255224    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:39:33.367007 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:33.366962    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podStartSLOduration=2.36694665 podStartE2EDuration="2.36694665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:39:33.364725046 +0000 UTC m=+1343.179516323" watchObservedRunningTime="2026-11-04 03:39:33.36694665 +0000 UTC m=+1343.181737981"
Nov 04 03:39:35.210591 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:35.210549    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:35.884237 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:35.884201    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:37.309347 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:37.309313    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:39:39.737491 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:39.737457    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:39.983880 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:39.963798    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:40.216920 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.214956    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x" podUID="f6339e78-6825-4246-a5d2-11831cd454af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 04 03:39:40.241303 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.241259    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf" podUID="70953f5d-4cbc-4cb5-8136-4045ee3af987" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:39:40.319868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.286886    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t" podUID="178304db-3ebd-4993-abd1-b283b7274d8a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 04 03:39:40.552902 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.523734    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg" podUID="5163cb68-1d76-43af-8654-990a89eb7405" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:39:40.760868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.712799    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"]
Nov 04 03:39:40.760868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.718644    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"]
Nov 04 03:39:40.760868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.719482    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"
Nov 04 03:39:40.760868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.752273    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5" podUID="3b12d15f-129b-4121-85ac-c83c3581fcc2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:39:40.760868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.752608    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb" podUID="85dd0445-88e0-4c88-8d9a-14fbf6bc2656" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": context deadline exceeded"
Nov 04 03:39:40.816216 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.816119    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-951893\""
Nov 04 03:39:40.843043 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.837178    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-951893-dockercfg-6gdx2\""
Nov 04 03:39:40.857866 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.849696    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv\" (UID: \"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"
Nov 04 03:39:40.857866 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.850014    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddbtl\" (UniqueName: \"kubernetes.io/projected/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-kube-api-access-ddbtl\") pod \"payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv\" (UID: \"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"
Nov 04 03:39:40.956867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.953235    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv\" (UID: \"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"
Nov 04 03:39:40.956867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.953383    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbtl\" (UniqueName: \"kubernetes.io/projected/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-kube-api-access-ddbtl\") pod \"payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv\" (UID: \"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"
Nov 04 03:39:40.956867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:40.955194    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv\" (UID: \"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"
Nov 04 03:39:41.035985 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:41.035939    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh" podUID="b747b84b-0b77-4776-9c2e-e101a00f4c21" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:39:41.082134 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:41.080174    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddbtl\" (UniqueName: \"kubernetes.io/projected/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-kube-api-access-ddbtl\") pod \"payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv\" (UID: \"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"
Nov 04 03:39:41.218075 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:41.218041    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"
Nov 04 03:39:41.219345 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:41.219012    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x" podUID="f6339e78-6825-4246-a5d2-11831cd454af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:39:41.220065 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:41.220045    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:39:41.243397 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:41.241951    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf" podUID="70953f5d-4cbc-4cb5-8136-4045ee3af987" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 04 03:39:41.243397 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:41.242302    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:39:41.330867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:41.324106    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:39:41.637864 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:41.637821    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:39:41.828137 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:41.827554    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"]
Nov 04 03:39:42.207049 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.206953    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:42.351887 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.348885    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"]
Nov 04 03:39:42.353358 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.353330    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:42.358611 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.358579    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:42.404020 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.403834    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 03:39:42.404020 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.403946    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:39:42.404020 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.403994    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 03:39:42.404502 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.404477    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:39:42.410030 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.410000    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"]
Nov 04 03:39:42.449676 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.449643    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7a343d6b-mfgqx\" (UID: \"7996fc27-5167-4b43-b003-b4686898bbc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:42.449866 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.449719    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8ldn\" (UniqueName: \"kubernetes.io/projected/7996fc27-5167-4b43-b003-b4686898bbc9-kube-api-access-z8ldn\") pod \"maas-api-revoke-sub-7a343d6b-mfgqx\" (UID: \"7996fc27-5167-4b43-b003-b4686898bbc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:42.449866 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.449815    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7a343d6b-mfgqx\" (UID: \"7996fc27-5167-4b43-b003-b4686898bbc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:42.551579 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.551551    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7a343d6b-mfgqx\" (UID: \"7996fc27-5167-4b43-b003-b4686898bbc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:42.551757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.551619    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8ldn\" (UniqueName: \"kubernetes.io/projected/7996fc27-5167-4b43-b003-b4686898bbc9-kube-api-access-z8ldn\") pod \"maas-api-revoke-sub-7a343d6b-mfgqx\" (UID: \"7996fc27-5167-4b43-b003-b4686898bbc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:42.551757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.551648    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7a343d6b-mfgqx\" (UID: \"7996fc27-5167-4b43-b003-b4686898bbc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:42.552486 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.552454    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7a343d6b-mfgqx\" (UID: \"7996fc27-5167-4b43-b003-b4686898bbc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:42.554218 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.554187    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7a343d6b-mfgqx\" (UID: \"7996fc27-5167-4b43-b003-b4686898bbc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:42.611352 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.611325    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8ldn\" (UniqueName: \"kubernetes.io/projected/7996fc27-5167-4b43-b003-b4686898bbc9-kube-api-access-z8ldn\") pod \"maas-api-revoke-sub-7a343d6b-mfgqx\" (UID: \"7996fc27-5167-4b43-b003-b4686898bbc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:42.668860 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.668813    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:42.697119 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.697078    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv" event={"ID":"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd","Type":"ContainerStarted","Data":"6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98"}
Nov 04 03:39:42.697264 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.697151    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv" event={"ID":"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd","Type":"ContainerStarted","Data":"719accd26a892947d41c2bd1740fb5219e94412916bbaa6c8fe0739905dff85a"}
Nov 04 03:39:42.697264 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.697181    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"
Nov 04 03:39:42.814721 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:42.814630    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv" podStartSLOduration=2.81461562 podStartE2EDuration="2.81461562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:39:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:39:42.813587749 +0000 UTC m=+1352.628379037" watchObservedRunningTime="2026-11-04 03:39:42.81461562 +0000 UTC m=+1352.629406932"
Nov 04 03:39:43.042224 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.042187    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"]
Nov 04 03:39:43.048220 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:39:43.048188    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7996fc27_5167_4b43_b003_b4686898bbc9.slice/crio-3fddb872276a036cc645b397768f39861acf403e988cd6b3181a6ece522cdd91 WatchSource:0}: Error finding container 3fddb872276a036cc645b397768f39861acf403e988cd6b3181a6ece522cdd91: Status 404 returned error can't find the container with id 3fddb872276a036cc645b397768f39861acf403e988cd6b3181a6ece522cdd91
Nov 04 03:39:43.258793 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.258771    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:39:43.575235 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.575203    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh" podUID="b747b84b-0b77-4776-9c2e-e101a00f4c21" containerName="main" containerID="cri-o://6b6ab39e42213c23124f2a3594c196f894a6723383b7e3bd95d5a1c2f87e707b" gracePeriod=45
Nov 04 03:39:43.705065 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.705034    4420 generic.go:356] "Generic (PLEG): container finished" podID="b747b84b-0b77-4776-9c2e-e101a00f4c21" containerID="6b6ab39e42213c23124f2a3594c196f894a6723383b7e3bd95d5a1c2f87e707b" exitCode=0
Nov 04 03:39:43.705233 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.705104    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh" event={"ID":"b747b84b-0b77-4776-9c2e-e101a00f4c21","Type":"ContainerDied","Data":"6b6ab39e42213c23124f2a3594c196f894a6723383b7e3bd95d5a1c2f87e707b"}
Nov 04 03:39:43.706857 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.706819    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx" event={"ID":"7996fc27-5167-4b43-b003-b4686898bbc9","Type":"ContainerStarted","Data":"e905d9d844b406853fcb01ff11a6d2d737ce77c764fccde65ceb461107623b28"}
Nov 04 03:39:43.706978 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.706919    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx" event={"ID":"7996fc27-5167-4b43-b003-b4686898bbc9","Type":"ContainerStarted","Data":"3fddb872276a036cc645b397768f39861acf403e988cd6b3181a6ece522cdd91"}
Nov 04 03:39:43.818809 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.818755    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx" podStartSLOduration=1.818741536 podStartE2EDuration="1.818741536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:39:43.815961966 +0000 UTC m=+1353.630753269" watchObservedRunningTime="2026-11-04 03:39:43.818741536 +0000 UTC m=+1353.633532813"
Nov 04 03:39:43.846217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.846193    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:39:43.849766 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.849735    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:43.869914 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.869888    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-dshm\") pod \"b747b84b-0b77-4776-9c2e-e101a00f4c21\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") "
Nov 04 03:39:43.870004 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.869947    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-home\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-home\") pod \"b747b84b-0b77-4776-9c2e-e101a00f4c21\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") "
Nov 04 03:39:43.870004 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.869974    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-tmp-dir\") pod \"b747b84b-0b77-4776-9c2e-e101a00f4c21\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") "
Nov 04 03:39:43.870004 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.870001    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-model-cache\") pod \"b747b84b-0b77-4776-9c2e-e101a00f4c21\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") "
Nov 04 03:39:43.870257 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.870031    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b747b84b-0b77-4776-9c2e-e101a00f4c21-kube-api-access-87t4v\" (UniqueName: \"kubernetes.io/projected/b747b84b-0b77-4776-9c2e-e101a00f4c21-kube-api-access-87t4v\") pod \"b747b84b-0b77-4776-9c2e-e101a00f4c21\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") "
Nov 04 03:39:43.870257 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.870078    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b747b84b-0b77-4776-9c2e-e101a00f4c21-tls-certs\" (UniqueName: \"kubernetes.io/secret/b747b84b-0b77-4776-9c2e-e101a00f4c21-tls-certs\") pod \"b747b84b-0b77-4776-9c2e-e101a00f4c21\" (UID: \"b747b84b-0b77-4776-9c2e-e101a00f4c21\") "
Nov 04 03:39:43.870355 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.870250    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-home" pod "b747b84b-0b77-4776-9c2e-e101a00f4c21" (UID: "b747b84b-0b77-4776-9c2e-e101a00f4c21"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:43.870355 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.870265    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-model-cache" pod "b747b84b-0b77-4776-9c2e-e101a00f4c21" (UID: "b747b84b-0b77-4776-9c2e-e101a00f4c21"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:43.870355 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.870279    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-tmp-dir" pod "b747b84b-0b77-4776-9c2e-e101a00f4c21" (UID: "b747b84b-0b77-4776-9c2e-e101a00f4c21"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:43.870587 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.870568    4420 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-home\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:43.870644 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.870591    4420 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-tmp-dir\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:43.870644 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.870606    4420 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-model-cache\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:43.872158 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.872129    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b747b84b-0b77-4776-9c2e-e101a00f4c21-tls-certs" pod "b747b84b-0b77-4776-9c2e-e101a00f4c21" (UID: "b747b84b-0b77-4776-9c2e-e101a00f4c21"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:39:43.872256 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.872155    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-dshm" pod "b747b84b-0b77-4776-9c2e-e101a00f4c21" (UID: "b747b84b-0b77-4776-9c2e-e101a00f4c21"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:39:43.872256 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.872226    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b747b84b-0b77-4776-9c2e-e101a00f4c21-kube-api-access-87t4v" pod "b747b84b-0b77-4776-9c2e-e101a00f4c21" (UID: "b747b84b-0b77-4776-9c2e-e101a00f4c21"). InnerVolumeSpecName "kube-api-access-87t4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:39:43.978727 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.978285    4420 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b747b84b-0b77-4776-9c2e-e101a00f4c21-dshm\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:43.978727 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.978323    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87t4v\" (UniqueName: \"kubernetes.io/projected/b747b84b-0b77-4776-9c2e-e101a00f4c21-kube-api-access-87t4v\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:43.978727 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:43.978340    4420 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b747b84b-0b77-4776-9c2e-e101a00f4c21-tls-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:44.052577 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:44.052545    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:44.263444 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:44.263408    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:39:44.713785 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:44.713753    4420 generic.go:356] "Generic (PLEG): container finished" podID="7996fc27-5167-4b43-b003-b4686898bbc9" containerID="e905d9d844b406853fcb01ff11a6d2d737ce77c764fccde65ceb461107623b28" exitCode=0
Nov 04 03:39:44.713966 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:44.713863    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx" event={"ID":"7996fc27-5167-4b43-b003-b4686898bbc9","Type":"ContainerDied","Data":"e905d9d844b406853fcb01ff11a6d2d737ce77c764fccde65ceb461107623b28"}
Nov 04 03:39:44.715200 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:44.715179    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh" event={"ID":"b747b84b-0b77-4776-9c2e-e101a00f4c21","Type":"ContainerDied","Data":"53ed6749d8613cd3d3fe58c60ce32a349176011eb45e0cfb62ac86a583dc0f6f"}
Nov 04 03:39:44.715200 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:44.715196    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"
Nov 04 03:39:44.715324 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:44.715216    4420 scope.go:122] "RemoveContainer" containerID="6b6ab39e42213c23124f2a3594c196f894a6723383b7e3bd95d5a1c2f87e707b"
Nov 04 03:39:44.934917 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:44.934207    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"]
Nov 04 03:39:44.946469 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:44.943214    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-ee1947/simulator-w1-ee1947-kserve-84bdfc9fff-kgbrh"]
Nov 04 03:39:45.879724 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:45.879694    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:46.000500 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.000473    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7996fc27-5167-4b43-b003-b4686898bbc9-kube-api-access-z8ldn\" (UniqueName: \"kubernetes.io/projected/7996fc27-5167-4b43-b003-b4686898bbc9-kube-api-access-z8ldn\") pod \"7996fc27-5167-4b43-b003-b4686898bbc9\" (UID: \"7996fc27-5167-4b43-b003-b4686898bbc9\") "
Nov 04 03:39:46.000673 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.000593    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-xks-ca\") pod \"7996fc27-5167-4b43-b003-b4686898bbc9\" (UID: \"7996fc27-5167-4b43-b003-b4686898bbc9\") "
Nov 04 03:39:46.000673 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.000640    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-openshift-ca\") pod \"7996fc27-5167-4b43-b003-b4686898bbc9\" (UID: \"7996fc27-5167-4b43-b003-b4686898bbc9\") "
Nov 04 03:39:46.001055 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.001025    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-openshift-ca" pod "7996fc27-5167-4b43-b003-b4686898bbc9" (UID: "7996fc27-5167-4b43-b003-b4686898bbc9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:39:46.002693 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.002673    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7996fc27-5167-4b43-b003-b4686898bbc9-kube-api-access-z8ldn" pod "7996fc27-5167-4b43-b003-b4686898bbc9" (UID: "7996fc27-5167-4b43-b003-b4686898bbc9"). InnerVolumeSpecName "kube-api-access-z8ldn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:39:46.003176 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.003155    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-xks-ca" pod "7996fc27-5167-4b43-b003-b4686898bbc9" (UID: "7996fc27-5167-4b43-b003-b4686898bbc9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:39:46.102618 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.102590    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-openshift-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:46.102618 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.102617    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8ldn\" (UniqueName: \"kubernetes.io/projected/7996fc27-5167-4b43-b003-b4686898bbc9-kube-api-access-z8ldn\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:46.102774 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.102626    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7996fc27-5167-4b43-b003-b4686898bbc9-maas-api-xks-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:39:46.306613 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.306541    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b747b84b-0b77-4776-9c2e-e101a00f4c21" path="/var/lib/kubelet/pods/b747b84b-0b77-4776-9c2e-e101a00f4c21/volumes"
Nov 04 03:39:46.728039 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.727965    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"
Nov 04 03:39:46.728039 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.727985    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx" event={"ID":"7996fc27-5167-4b43-b003-b4686898bbc9","Type":"ContainerDied","Data":"3fddb872276a036cc645b397768f39861acf403e988cd6b3181a6ece522cdd91"}
Nov 04 03:39:46.728039 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:46.728031    4420 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fddb872276a036cc645b397768f39861acf403e988cd6b3181a6ece522cdd91"
Nov 04 03:39:47.687570 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:47.687528    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:47.890383 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:47.890342    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:48.740032 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:48.739989    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:52.364644 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:52.364612    4420 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-243e19a17a73: secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:52.365209 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:52.365184    4420 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d: secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:52.365334 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:52.365286    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials podName:8165ded7-cbdd-41e6-9790-06ad19ab9a2e nodeName:}" failed. No retries permitted until 2026-11-04 03:39:52.86526544 +0000 UTC m=+1362.680056719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials") pod "payload-processing-external-model-7f5c68695b-7z22d" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e") : secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:52.834621 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:52.834586    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:52.869480 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:52.869453    4420 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-243e19a17a73: secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:52.869480 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:52.869479    4420 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d: secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:52.869643 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:52.869535    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials podName:8165ded7-cbdd-41e6-9790-06ad19ab9a2e nodeName:}" failed. No retries permitted until 2026-11-04 03:39:53.869520473 +0000 UTC m=+1363.684311728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials") pod "payload-processing-external-model-7f5c68695b-7z22d" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e") : secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:53.710204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:53.710178    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"
Nov 04 03:39:53.879329 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:53.879288    4420 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-243e19a17a73: secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:53.879329 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:53.879327    4420 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d: secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:53.879530 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:53.879404    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials podName:8165ded7-cbdd-41e6-9790-06ad19ab9a2e nodeName:}" failed. No retries permitted until 2026-11-04 03:39:55.879384356 +0000 UTC m=+1365.694175633 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials") pod "payload-processing-external-model-7f5c68695b-7z22d" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e") : secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:54.262423 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:54.262386    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:39:54.601551 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:54.601472    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:55.900563 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:55.900529    4420 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-243e19a17a73: secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:55.900563 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:55.900563    4420 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d: secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:55.901114 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:55.900620    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials podName:8165ded7-cbdd-41e6-9790-06ad19ab9a2e nodeName:}" failed. No retries permitted until 2026-11-04 03:39:59.900605783 +0000 UTC m=+1369.715397038 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials") pod "payload-processing-external-model-7f5c68695b-7z22d" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e") : secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:57.734413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:57.734381    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:57.970180 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:57.970145    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:58.285307 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:58.285275    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:58.454354 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:58.454319    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:58.599006 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:58.598932    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:59.327169 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:59.327130    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:59.523318 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:39:59.523283    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:39:59.942467 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:59.942437    4420 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-243e19a17a73: secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:59.942467 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:59.942466    4420 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d: secret "agc-cred-243e19a17a73" not found
Nov 04 03:39:59.942673 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:39:59.942518    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials podName:8165ded7-cbdd-41e6-9790-06ad19ab9a2e nodeName:}" failed. No retries permitted until 2026-11-04 03:40:07.942504944 +0000 UTC m=+1377.757296200 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials") pod "payload-processing-external-model-7f5c68695b-7z22d" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e") : secret "agc-cred-243e19a17a73" not found
Nov 04 03:40:02.013642 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:02.013600    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:02.652045 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:02.652009    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:40:02.957043 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:02.956958    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:04.262271 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:04.262235    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:40:05.120701 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:05.120663    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:05.267218 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:05.267176    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:06.659284 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:06.659252    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:07.391612 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:07.391577    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:07.585172 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:07.585135    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:07.827613 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:07.827577    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:08.033311 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:08.033276    4420 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-243e19a17a73: secret "agc-cred-243e19a17a73" not found
Nov 04 03:40:08.033311 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:08.033304    4420 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d: secret "agc-cred-243e19a17a73" not found
Nov 04 03:40:08.033522 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:08.033372    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials podName:8165ded7-cbdd-41e6-9790-06ad19ab9a2e nodeName:}" failed. No retries permitted until 2026-11-04 03:40:24.033355525 +0000 UTC m=+1393.848146780 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials") pod "payload-processing-external-model-7f5c68695b-7z22d" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e") : secret "agc-cred-243e19a17a73" not found
Nov 04 03:40:12.334452 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:12.334413    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:12.484192 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:12.484146    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:12.828157 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:12.828125    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:14.262351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:14.262317    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:40:17.247605 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:17.247571    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:18.535973 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:18.535941    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:18.774532 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:18.774502    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:18.941419 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:18.941334    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:19.168953 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:19.168924    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:19.937028 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:19.936994    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:20.373666 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:20.373617    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:20.992989 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:20.992958    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"]
Nov 04 03:40:20.993385 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:20.993174    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" podUID="114cc480-e80a-41d7-bd80-ce73ceebc155" containerName="istio-proxy" containerID="cri-o://70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70" gracePeriod=30
Nov 04 03:40:21.011302 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:21.011272    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:21.039777 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:21.039738    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"]
Nov 04 03:40:21.040017 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:21.039990    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" podUID="e107aafd-b91e-4a9f-8512-66744aec6a95" containerName="istio-proxy" containerID="cri-o://65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4" gracePeriod=30
Nov 04 03:40:21.042202 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:21.042177    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"]
Nov 04 03:40:21.042357 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:21.042331    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz" podUID="656d246c-97d1-46e3-881c-a31c4a0eb32e" containerName="istio-proxy" containerID="cri-o://bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec" gracePeriod=30
Nov 04 03:40:22.520078 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:22.520039    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:40:22.520560 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:22.520117    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" podUID="e107aafd-b91e-4a9f-8512-66744aec6a95" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:40:22.565034 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:22.565005    4420 patch_prober.go:29] interesting pod/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:40:22.565186 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:22.565077    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" podUID="114cc480-e80a-41d7-bd80-ce73ceebc155" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:40:24.101907 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:24.101863    4420 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-243e19a17a73: secret "agc-cred-243e19a17a73" not found
Nov 04 03:40:24.101907 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:24.101897    4420 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d: secret "agc-cred-243e19a17a73" not found
Nov 04 03:40:24.102329 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:24.101951    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials podName:8165ded7-cbdd-41e6-9790-06ad19ab9a2e nodeName:}" failed. No retries permitted until 2026-11-04 03:40:56.101937612 +0000 UTC m=+1425.916728867 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials") pod "payload-processing-external-model-7f5c68695b-7z22d" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e") : secret "agc-cred-243e19a17a73" not found
Nov 04 03:40:24.262290 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:24.262255    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:40:24.451568 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:24.451475    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:24.652651 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:24.652619    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:24.802868 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:24.802801    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:26.312080 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.311931    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:40:26.365946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.365921    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:40:26.370018 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.369996    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:40:26.427598 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.427561    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-data\") pod \"656d246c-97d1-46e3-881c-a31c4a0eb32e\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") "
Nov 04 03:40:26.427598 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.427618    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/656d246c-97d1-46e3-881c-a31c4a0eb32e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/656d246c-97d1-46e3-881c-a31c4a0eb32e-istiod-ca-cert\") pod \"656d246c-97d1-46e3-881c-a31c4a0eb32e\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") "
Nov 04 03:40:26.427988 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.427662    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-token\" (UniqueName: \"kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-token\") pod \"114cc480-e80a-41d7-bd80-ce73ceebc155\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") "
Nov 04 03:40:26.427988 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.427695    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e107aafd-b91e-4a9f-8512-66744aec6a95-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e107aafd-b91e-4a9f-8512-66744aec6a95-istiod-ca-cert\") pod \"e107aafd-b91e-4a9f-8512-66744aec6a95\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") "
Nov 04 03:40:26.427988 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.427752    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-envoy\") pod \"e107aafd-b91e-4a9f-8512-66744aec6a95\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") "
Nov 04 03:40:26.427988 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.427797    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-certs\") pod \"656d246c-97d1-46e3-881c-a31c4a0eb32e\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") "
Nov 04 03:40:26.427988 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.427824    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-kube-api-access-jn7rm\" (UniqueName: \"kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-kube-api-access-jn7rm\") pod \"114cc480-e80a-41d7-bd80-ce73ceebc155\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") "
Nov 04 03:40:26.427988 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.427870    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-envoy\") pod \"114cc480-e80a-41d7-bd80-ce73ceebc155\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") "
Nov 04 03:40:26.427988 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.427922    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-podinfo\") pod \"114cc480-e80a-41d7-bd80-ce73ceebc155\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") "
Nov 04 03:40:26.427988 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.427957    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-certs\") pod \"114cc480-e80a-41d7-bd80-ce73ceebc155\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") "
Nov 04 03:40:26.427988 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.427985    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-credential-socket\") pod \"656d246c-97d1-46e3-881c-a31c4a0eb32e\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") "
Nov 04 03:40:26.428399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428018    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-token\" (UniqueName: \"kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-token\") pod \"656d246c-97d1-46e3-881c-a31c4a0eb32e\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") "
Nov 04 03:40:26.428399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428052    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-kube-api-access-vbb2x\" (UniqueName: \"kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-kube-api-access-vbb2x\") pod \"e107aafd-b91e-4a9f-8512-66744aec6a95\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") "
Nov 04 03:40:26.428399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428079    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-data\") pod \"114cc480-e80a-41d7-bd80-ce73ceebc155\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") "
Nov 04 03:40:26.428399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428110    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/114cc480-e80a-41d7-bd80-ce73ceebc155-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/114cc480-e80a-41d7-bd80-ce73ceebc155-istiod-ca-cert\") pod \"114cc480-e80a-41d7-bd80-ce73ceebc155\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") "
Nov 04 03:40:26.428399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428145    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-podinfo\") pod \"e107aafd-b91e-4a9f-8512-66744aec6a95\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") "
Nov 04 03:40:26.428399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428179    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-credential-socket\") pod \"e107aafd-b91e-4a9f-8512-66744aec6a95\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") "
Nov 04 03:40:26.428399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428300    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-token\" (UniqueName: \"kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-token\") pod \"e107aafd-b91e-4a9f-8512-66744aec6a95\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") "
Nov 04 03:40:26.428399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428308    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-data" pod "656d246c-97d1-46e3-881c-a31c4a0eb32e" (UID: "656d246c-97d1-46e3-881c-a31c4a0eb32e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.428399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428336    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-socket\") pod \"e107aafd-b91e-4a9f-8512-66744aec6a95\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") "
Nov 04 03:40:26.428399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428395    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-certs\") pod \"e107aafd-b91e-4a9f-8512-66744aec6a95\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") "
Nov 04 03:40:26.428911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428431    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-envoy\") pod \"656d246c-97d1-46e3-881c-a31c4a0eb32e\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") "
Nov 04 03:40:26.428911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428473    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-socket\") pod \"656d246c-97d1-46e3-881c-a31c4a0eb32e\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") "
Nov 04 03:40:26.428911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428510    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-credential-socket\") pod \"114cc480-e80a-41d7-bd80-ce73ceebc155\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") "
Nov 04 03:40:26.428911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428565    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-kube-api-access-vws9n\" (UniqueName: \"kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-kube-api-access-vws9n\") pod \"656d246c-97d1-46e3-881c-a31c4a0eb32e\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") "
Nov 04 03:40:26.428911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428603    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-data\") pod \"e107aafd-b91e-4a9f-8512-66744aec6a95\" (UID: \"e107aafd-b91e-4a9f-8512-66744aec6a95\") "
Nov 04 03:40:26.428911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428641    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-socket\") pod \"114cc480-e80a-41d7-bd80-ce73ceebc155\" (UID: \"114cc480-e80a-41d7-bd80-ce73ceebc155\") "
Nov 04 03:40:26.428911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.428719    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-podinfo\") pod \"656d246c-97d1-46e3-881c-a31c4a0eb32e\" (UID: \"656d246c-97d1-46e3-881c-a31c4a0eb32e\") "
Nov 04 03:40:26.430927 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.429541    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-certs" pod "114cc480-e80a-41d7-bd80-ce73ceebc155" (UID: "114cc480-e80a-41d7-bd80-ce73ceebc155"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.430927 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.429628    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/114cc480-e80a-41d7-bd80-ce73ceebc155-istiod-ca-cert" pod "114cc480-e80a-41d7-bd80-ce73ceebc155" (UID: "114cc480-e80a-41d7-bd80-ce73ceebc155"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:26.430927 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.429793    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-certs" pod "656d246c-97d1-46e3-881c-a31c4a0eb32e" (UID: "656d246c-97d1-46e3-881c-a31c4a0eb32e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.430927 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.429927    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-credential-socket" pod "656d246c-97d1-46e3-881c-a31c4a0eb32e" (UID: "656d246c-97d1-46e3-881c-a31c4a0eb32e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.430927 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.430281    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-socket" pod "656d246c-97d1-46e3-881c-a31c4a0eb32e" (UID: "656d246c-97d1-46e3-881c-a31c4a0eb32e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.430927 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.430455    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-credential-socket" pod "114cc480-e80a-41d7-bd80-ce73ceebc155" (UID: "114cc480-e80a-41d7-bd80-ce73ceebc155"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.430927 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.430622    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e107aafd-b91e-4a9f-8512-66744aec6a95-istiod-ca-cert" pod "e107aafd-b91e-4a9f-8512-66744aec6a95" (UID: "e107aafd-b91e-4a9f-8512-66744aec6a95"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:26.431356 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.430929    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-data" pod "114cc480-e80a-41d7-bd80-ce73ceebc155" (UID: "114cc480-e80a-41d7-bd80-ce73ceebc155"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.431356 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.431304    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/656d246c-97d1-46e3-881c-a31c4a0eb32e-istiod-ca-cert" pod "656d246c-97d1-46e3-881c-a31c4a0eb32e" (UID: "656d246c-97d1-46e3-881c-a31c4a0eb32e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:26.433341 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.432890    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-socket" pod "e107aafd-b91e-4a9f-8512-66744aec6a95" (UID: "e107aafd-b91e-4a9f-8512-66744aec6a95"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.433341 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.433112    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-certs" pod "e107aafd-b91e-4a9f-8512-66744aec6a95" (UID: "e107aafd-b91e-4a9f-8512-66744aec6a95"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.435339 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.435140    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-envoy" pod "656d246c-97d1-46e3-881c-a31c4a0eb32e" (UID: "656d246c-97d1-46e3-881c-a31c4a0eb32e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.435339 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.435161    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-token" pod "656d246c-97d1-46e3-881c-a31c4a0eb32e" (UID: "656d246c-97d1-46e3-881c-a31c4a0eb32e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.435339 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.435321    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-kube-api-access-vws9n" pod "656d246c-97d1-46e3-881c-a31c4a0eb32e" (UID: "656d246c-97d1-46e3-881c-a31c4a0eb32e"). InnerVolumeSpecName "kube-api-access-vws9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.435545 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.435531    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-token" pod "e107aafd-b91e-4a9f-8512-66744aec6a95" (UID: "e107aafd-b91e-4a9f-8512-66744aec6a95"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.435813 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.435639    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-socket" pod "114cc480-e80a-41d7-bd80-ce73ceebc155" (UID: "114cc480-e80a-41d7-bd80-ce73ceebc155"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.435974    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-data" pod "e107aafd-b91e-4a9f-8512-66744aec6a95" (UID: "e107aafd-b91e-4a9f-8512-66744aec6a95"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436034    4420 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e107aafd-b91e-4a9f-8512-66744aec6a95-istiod-ca-cert\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436061    4420 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436077    4420 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436096    4420 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-credential-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436112    4420 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-token\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436127    4420 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-data\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436143    4420 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/114cc480-e80a-41d7-bd80-ce73ceebc155-istiod-ca-cert\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436186    4420 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-token\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436202    4420 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436216    4420 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-workload-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436231    4420 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-envoy\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436249 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436247    4420 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-workload-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436990 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436261    4420 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-credential-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436990 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436281    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vws9n\" (UniqueName: \"kubernetes.io/projected/656d246c-97d1-46e3-881c-a31c4a0eb32e-kube-api-access-vws9n\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436990 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436297    4420 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-workload-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436990 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436311    4420 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-data\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436990 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436325    4420 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/656d246c-97d1-46e3-881c-a31c4a0eb32e-istiod-ca-cert\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.436990 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436581    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-kube-api-access-jn7rm" pod "114cc480-e80a-41d7-bd80-ce73ceebc155" (UID: "114cc480-e80a-41d7-bd80-ce73ceebc155"). InnerVolumeSpecName "kube-api-access-jn7rm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.436990 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.436680    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-credential-socket" pod "e107aafd-b91e-4a9f-8512-66744aec6a95" (UID: "e107aafd-b91e-4a9f-8512-66744aec6a95"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.437357 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.437330    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-podinfo" pod "656d246c-97d1-46e3-881c-a31c4a0eb32e" (UID: "656d246c-97d1-46e3-881c-a31c4a0eb32e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:40:26.437695 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.437680    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-podinfo" pod "e107aafd-b91e-4a9f-8512-66744aec6a95" (UID: "e107aafd-b91e-4a9f-8512-66744aec6a95"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:40:26.439037 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.439016    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-kube-api-access-vbb2x" pod "e107aafd-b91e-4a9f-8512-66744aec6a95" (UID: "e107aafd-b91e-4a9f-8512-66744aec6a95"). InnerVolumeSpecName "kube-api-access-vbb2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.439306 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.439284    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-token" pod "114cc480-e80a-41d7-bd80-ce73ceebc155" (UID: "114cc480-e80a-41d7-bd80-ce73ceebc155"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:26.439306 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.439299    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-envoy" pod "e107aafd-b91e-4a9f-8512-66744aec6a95" (UID: "e107aafd-b91e-4a9f-8512-66744aec6a95"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.439549 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.439521    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-envoy" pod "114cc480-e80a-41d7-bd80-ce73ceebc155" (UID: "114cc480-e80a-41d7-bd80-ce73ceebc155"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:40:26.440838 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.440816    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-podinfo" pod "114cc480-e80a-41d7-bd80-ce73ceebc155" (UID: "114cc480-e80a-41d7-bd80-ce73ceebc155"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:40:26.537857 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.537759    4420 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-token\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.537857 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.537786    4420 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-envoy\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.537857 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.537796    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn7rm\" (UniqueName: \"kubernetes.io/projected/114cc480-e80a-41d7-bd80-ce73ceebc155-kube-api-access-jn7rm\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.537857 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.537804    4420 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-envoy\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.537857 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.537813    4420 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/114cc480-e80a-41d7-bd80-ce73ceebc155-istio-podinfo\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.537857 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.537822    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbb2x\" (UniqueName: \"kubernetes.io/projected/e107aafd-b91e-4a9f-8512-66744aec6a95-kube-api-access-vbb2x\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.537857 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.537830    4420 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-podinfo\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.537857 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.537852    4420 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-credential-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.537857 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.537861    4420 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e107aafd-b91e-4a9f-8512-66744aec6a95-istio-data\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.538416 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.537869    4420 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/656d246c-97d1-46e3-881c-a31c4a0eb32e-istio-podinfo\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:26.944689 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.944606    4420 generic.go:356] "Generic (PLEG): container finished" podID="114cc480-e80a-41d7-bd80-ce73ceebc155" containerID="70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70" exitCode=0
Nov 04 03:40:26.944689 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.944663    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"
Nov 04 03:40:26.944942 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.944691    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" event={"ID":"114cc480-e80a-41d7-bd80-ce73ceebc155","Type":"ContainerDied","Data":"70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70"}
Nov 04 03:40:26.944942 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.944780    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8" event={"ID":"114cc480-e80a-41d7-bd80-ce73ceebc155","Type":"ContainerDied","Data":"a1f0e4b90ab9cceb8ad5125b666f312a857c4f571f798488a955f1f62ca00ebe"}
Nov 04 03:40:26.944942 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.944811    4420 scope.go:122] "RemoveContainer" containerID="70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70"
Nov 04 03:40:26.946407 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.946380    4420 generic.go:356] "Generic (PLEG): container finished" podID="656d246c-97d1-46e3-881c-a31c4a0eb32e" containerID="bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec" exitCode=0
Nov 04 03:40:26.946519 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.946431    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"
Nov 04 03:40:26.946519 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.946470    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz" event={"ID":"656d246c-97d1-46e3-881c-a31c4a0eb32e","Type":"ContainerDied","Data":"bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec"}
Nov 04 03:40:26.946635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.946527    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz" event={"ID":"656d246c-97d1-46e3-881c-a31c4a0eb32e","Type":"ContainerDied","Data":"176aecdfff00cb2be42c467df3bab3ad30ca5fdf0b29565a9641900cc9d3cb97"}
Nov 04 03:40:26.948071 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.948017    4420 generic.go:356] "Generic (PLEG): container finished" podID="e107aafd-b91e-4a9f-8512-66744aec6a95" containerID="65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4" exitCode=0
Nov 04 03:40:26.948198 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.948083    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" event={"ID":"e107aafd-b91e-4a9f-8512-66744aec6a95","Type":"ContainerDied","Data":"65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4"}
Nov 04 03:40:26.948198 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.948120    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh" event={"ID":"e107aafd-b91e-4a9f-8512-66744aec6a95","Type":"ContainerDied","Data":"9c367ea008e2014bed02a61041f8b30704bfb4b547b6c9331f2d19d2c1026227"}
Nov 04 03:40:26.948198 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.948194    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"
Nov 04 03:40:26.954569 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.954506    4420 scope.go:122] "RemoveContainer" containerID="70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70"
Nov 04 03:40:26.954876 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:26.954824    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70\": container with ID starting with 70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70 not found: ID does not exist" containerID="70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70"
Nov 04 03:40:26.954984 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.954884    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70"} err="failed to get container status \"70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70\": rpc error: code = NotFound desc = could not find container \"70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70\": container with ID starting with 70944bec1a82ed42e92e4826a3412c729bbd95d141cda9b030f53e441b9bec70 not found: ID does not exist"
Nov 04 03:40:26.954984 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.954904    4420 scope.go:122] "RemoveContainer" containerID="bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec"
Nov 04 03:40:26.963496 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.963481    4420 scope.go:122] "RemoveContainer" containerID="bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec"
Nov 04 03:40:26.963738 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:26.963719    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec\": container with ID starting with bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec not found: ID does not exist" containerID="bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec"
Nov 04 03:40:26.963801 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.963747    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec"} err="failed to get container status \"bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec\": rpc error: code = NotFound desc = could not find container \"bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec\": container with ID starting with bfeda732994d9afacfe19675efb02397b3fc23f06d53093948edee1694bb02ec not found: ID does not exist"
Nov 04 03:40:26.963801 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.963768    4420 scope.go:122] "RemoveContainer" containerID="65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4"
Nov 04 03:40:26.971347 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.971331    4420 scope.go:122] "RemoveContainer" containerID="65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4"
Nov 04 03:40:26.971606 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:26.971589    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4\": container with ID starting with 65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4 not found: ID does not exist" containerID="65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4"
Nov 04 03:40:26.971673 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:26.971610    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4"} err="failed to get container status \"65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4\": rpc error: code = NotFound desc = could not find container \"65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4\": container with ID starting with 65ef8a3becc96b9f6a15842df78ebea6bd41157809b59d1144b1ecbd27ddfee4 not found: ID does not exist"
Nov 04 03:40:27.099921 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:27.099882    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"]
Nov 04 03:40:27.146517 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:27.146487    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-9mzkh"]
Nov 04 03:40:27.209188 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:27.209126    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"]
Nov 04 03:40:27.213397 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:27.213377    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rwndz"]
Nov 04 03:40:27.316426 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:27.316395    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"]
Nov 04 03:40:27.325566 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:27.325536    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ee1947-openshift-default-7f6dcb4f6f-rjvs8"]
Nov 04 03:40:28.306756 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:28.306723    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="114cc480-e80a-41d7-bd80-ce73ceebc155" path="/var/lib/kubelet/pods/114cc480-e80a-41d7-bd80-ce73ceebc155/volumes"
Nov 04 03:40:28.307226 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:28.307210    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="656d246c-97d1-46e3-881c-a31c4a0eb32e" path="/var/lib/kubelet/pods/656d246c-97d1-46e3-881c-a31c4a0eb32e/volumes"
Nov 04 03:40:28.307601 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:28.307588    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e107aafd-b91e-4a9f-8512-66744aec6a95" path="/var/lib/kubelet/pods/e107aafd-b91e-4a9f-8512-66744aec6a95/volumes"
Nov 04 03:40:30.590694 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:30.590654    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:32.650214 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:32.650176    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:40:34.261637 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:34.261592    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:40:34.332481 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:34.332452    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:38.376674 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:38.376642    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:40:39.250802 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:39.250764    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:44.269690 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:44.269644    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:40:44.333363 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:44.333327    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:49.108232 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.108196    4420 generic.go:356] "Generic (PLEG): container finished" podID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerID="b114c09d6992206d59be7de814dba1a468fb8634556a64da10f048e20d2e27e1" exitCode=0
Nov 04 03:40:49.108775 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.108277    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" event={"ID":"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e","Type":"ContainerDied","Data":"b114c09d6992206d59be7de814dba1a468fb8634556a64da10f048e20d2e27e1"}
Nov 04 03:40:49.108870 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.108779    4420 scope.go:122] "RemoveContainer" containerID="b114c09d6992206d59be7de814dba1a468fb8634556a64da10f048e20d2e27e1"
Nov 04 03:40:49.110246 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.110218    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:40:49.110316 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.110288    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:40:49.110352 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.110321    4420 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 04 03:40:49.224240 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.224200    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:49.583749 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.583717    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"]
Nov 04 03:40:49.584184 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584170    4420 cpu_manager.go:414] "containerMap: removing container" podUID="e107aafd-b91e-4a9f-8512-66744aec6a95" containerName="istio-proxy"
Nov 04 03:40:49.584233 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584186    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e107aafd-b91e-4a9f-8512-66744aec6a95" containerName="istio-proxy"
Nov 04 03:40:49.584233 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584202    4420 cpu_manager.go:414] "containerMap: removing container" podUID="656d246c-97d1-46e3-881c-a31c4a0eb32e" containerName="istio-proxy"
Nov 04 03:40:49.584233 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584208    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="656d246c-97d1-46e3-881c-a31c4a0eb32e" containerName="istio-proxy"
Nov 04 03:40:49.584233 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584214    4420 cpu_manager.go:414] "containerMap: removing container" podUID="114cc480-e80a-41d7-bd80-ce73ceebc155" containerName="istio-proxy"
Nov 04 03:40:49.584233 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584220    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="114cc480-e80a-41d7-bd80-ce73ceebc155" containerName="istio-proxy"
Nov 04 03:40:49.584233 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584231    4420 cpu_manager.go:414] "containerMap: removing container" podUID="7996fc27-5167-4b43-b003-b4686898bbc9" containerName="revoke-keys"
Nov 04 03:40:49.584413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584236    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7996fc27-5167-4b43-b003-b4686898bbc9" containerName="revoke-keys"
Nov 04 03:40:49.584413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584251    4420 cpu_manager.go:414] "containerMap: removing container" podUID="b747b84b-0b77-4776-9c2e-e101a00f4c21" containerName="main"
Nov 04 03:40:49.584413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584255    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b747b84b-0b77-4776-9c2e-e101a00f4c21" containerName="main"
Nov 04 03:40:49.584413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584329    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="b747b84b-0b77-4776-9c2e-e101a00f4c21" containerName="main"
Nov 04 03:40:49.584413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584340    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="e107aafd-b91e-4a9f-8512-66744aec6a95" containerName="istio-proxy"
Nov 04 03:40:49.584413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584347    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="656d246c-97d1-46e3-881c-a31c4a0eb32e" containerName="istio-proxy"
Nov 04 03:40:49.584413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584353    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="114cc480-e80a-41d7-bd80-ce73ceebc155" containerName="istio-proxy"
Nov 04 03:40:49.584413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.584359    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="7996fc27-5167-4b43-b003-b4686898bbc9" containerName="revoke-keys"
Nov 04 03:40:49.587338 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.587317    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:49.639243 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.639212    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 03:40:49.639469 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.639438    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:40:49.639589 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.639509    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 03:40:49.639702 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.639685    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:40:49.642155 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.642133    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"]
Nov 04 03:40:49.670629 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.670588    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbbfs\" (UniqueName: \"kubernetes.io/projected/7ad0f554-b218-4d84-9c72-6ca76a6361a7-kube-api-access-gbbfs\") pod \"maas-api-revoke-sub-ec3256c7-pd5p2\" (UID: \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:49.670832 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.670795    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec3256c7-pd5p2\" (UID: \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:49.670917 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.670829    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec3256c7-pd5p2\" (UID: \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:49.772077 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.772045    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec3256c7-pd5p2\" (UID: \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:49.772077 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.772081    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec3256c7-pd5p2\" (UID: \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:49.772301 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.772155    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbbfs\" (UniqueName: \"kubernetes.io/projected/7ad0f554-b218-4d84-9c72-6ca76a6361a7-kube-api-access-gbbfs\") pod \"maas-api-revoke-sub-ec3256c7-pd5p2\" (UID: \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:49.772664 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.772644    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec3256c7-pd5p2\" (UID: \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:49.774399 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.774380    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec3256c7-pd5p2\" (UID: \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:49.829615 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.829590    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbbfs\" (UniqueName: \"kubernetes.io/projected/7ad0f554-b218-4d84-9c72-6ca76a6361a7-kube-api-access-gbbfs\") pod \"maas-api-revoke-sub-ec3256c7-pd5p2\" (UID: \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:49.900260 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:49.900199    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:50.115602 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:50.115568    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" event={"ID":"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e","Type":"ContainerStarted","Data":"feaf5ad24eb85657a65733f4ebe275bc6ecf93113201c2585896bbdf3520f6a9"}
Nov 04 03:40:50.277961 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:50.277929    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"]
Nov 04 03:40:50.316247 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:40:50.316214    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad0f554_b218_4d84_9c72_6ca76a6361a7.slice/crio-453bb31a4b3fa35aec21a3f160c96dc4fae3012423340fc2449f0b49d5ec427c WatchSource:0}: Error finding container 453bb31a4b3fa35aec21a3f160c96dc4fae3012423340fc2449f0b49d5ec427c: Status 404 returned error can't find the container with id 453bb31a4b3fa35aec21a3f160c96dc4fae3012423340fc2449f0b49d5ec427c
Nov 04 03:40:50.510477 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:50.510445    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:40:50.510648 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:50.510598    4420 patch_prober.go:29] interesting pod/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 04 03:40:50.510648 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:50.510635    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" podUID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 04 03:40:51.122860 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:51.122813    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2" event={"ID":"7ad0f554-b218-4d84-9c72-6ca76a6361a7","Type":"ContainerStarted","Data":"8b3b65ce2cdf628deeaed36293d2611e1ac103788996b7f39026aa742a9eb660"}
Nov 04 03:40:51.123350 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:51.122884    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2" event={"ID":"7ad0f554-b218-4d84-9c72-6ca76a6361a7","Type":"ContainerStarted","Data":"453bb31a4b3fa35aec21a3f160c96dc4fae3012423340fc2449f0b49d5ec427c"}
Nov 04 03:40:51.233367 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:51.233309    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2" podStartSLOduration=2.233296816 podStartE2EDuration="2.233296816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:40:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:40:51.231485862 +0000 UTC m=+1421.046277139" watchObservedRunningTime="2026-11-04 03:40:51.233296816 +0000 UTC m=+1421.048088093"
Nov 04 03:40:51.510796 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:51.510759    4420 patch_prober.go:29] interesting pod/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 04 03:40:51.510983 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:51.510816    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" podUID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 04 03:40:52.130384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:52.130348    4420 generic.go:356] "Generic (PLEG): container finished" podID="7ad0f554-b218-4d84-9c72-6ca76a6361a7" containerID="8b3b65ce2cdf628deeaed36293d2611e1ac103788996b7f39026aa742a9eb660" exitCode=0
Nov 04 03:40:52.130767 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:52.130430    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2" event={"ID":"7ad0f554-b218-4d84-9c72-6ca76a6361a7","Type":"ContainerDied","Data":"8b3b65ce2cdf628deeaed36293d2611e1ac103788996b7f39026aa742a9eb660"}
Nov 04 03:40:52.510605 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:52.510568    4420 patch_prober.go:29] interesting pod/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 04 03:40:52.510779 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:52.510631    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" podUID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 04 03:40:53.311786 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:53.311765    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:53.412773 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:53.412681    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-xks-ca\") pod \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\" (UID: \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\") "
Nov 04 03:40:53.412968 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:53.412804    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ad0f554-b218-4d84-9c72-6ca76a6361a7-kube-api-access-gbbfs\" (UniqueName: \"kubernetes.io/projected/7ad0f554-b218-4d84-9c72-6ca76a6361a7-kube-api-access-gbbfs\") pod \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\" (UID: \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\") "
Nov 04 03:40:53.412968 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:53.412864    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-openshift-ca\") pod \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\" (UID: \"7ad0f554-b218-4d84-9c72-6ca76a6361a7\") "
Nov 04 03:40:53.413274 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:53.413197    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-openshift-ca" pod "7ad0f554-b218-4d84-9c72-6ca76a6361a7" (UID: "7ad0f554-b218-4d84-9c72-6ca76a6361a7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:40:53.415130 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:53.415096    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-xks-ca" pod "7ad0f554-b218-4d84-9c72-6ca76a6361a7" (UID: "7ad0f554-b218-4d84-9c72-6ca76a6361a7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:40:53.415211 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:53.415181    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad0f554-b218-4d84-9c72-6ca76a6361a7-kube-api-access-gbbfs" pod "7ad0f554-b218-4d84-9c72-6ca76a6361a7" (UID: "7ad0f554-b218-4d84-9c72-6ca76a6361a7"). InnerVolumeSpecName "kube-api-access-gbbfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:40:53.511192 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:53.511159    4420 patch_prober.go:29] interesting pod/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 04 03:40:53.511361 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:53.511214    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" podUID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 04 03:40:53.513745 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:53.513692    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-openshift-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:53.513745 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:53.513718    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ad0f554-b218-4d84-9c72-6ca76a6361a7-maas-api-xks-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:53.513745 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:53.513728    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbbfs\" (UniqueName: \"kubernetes.io/projected/7ad0f554-b218-4d84-9c72-6ca76a6361a7-kube-api-access-gbbfs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:40:54.105051 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:54.104956    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:54.141706 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:54.141678    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"
Nov 04 03:40:54.141706 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:54.141699    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2" event={"ID":"7ad0f554-b218-4d84-9c72-6ca76a6361a7","Type":"ContainerDied","Data":"453bb31a4b3fa35aec21a3f160c96dc4fae3012423340fc2449f0b49d5ec427c"}
Nov 04 03:40:54.141911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:54.141765    4420 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="453bb31a4b3fa35aec21a3f160c96dc4fae3012423340fc2449f0b49d5ec427c"
Nov 04 03:40:54.269995 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:54.269962    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:40:54.510911 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:54.510882    4420 patch_prober.go:29] interesting pod/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 04 03:40:54.511281 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:54.510931    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" podUID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 04 03:40:55.510644 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:55.510603    4420 patch_prober.go:29] interesting pod/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 04 03:40:55.510897 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:55.510662    4420 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" podUID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 04 03:40:56.141808 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:56.141778    4420 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-243e19a17a73: secret "agc-cred-243e19a17a73" not found
Nov 04 03:40:56.141808 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:56.141804    4420 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d: secret "agc-cred-243e19a17a73" not found
Nov 04 03:40:56.142262 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:40:56.141867    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials podName:8165ded7-cbdd-41e6-9790-06ad19ab9a2e nodeName:}" failed. No retries permitted until 2026-11-04 03:42:00.141839721 +0000 UTC m=+1489.956630977 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials") pod "payload-processing-external-model-7f5c68695b-7z22d" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e") : secret "agc-cred-243e19a17a73" not found
Nov 04 03:40:56.515176 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:56.515144    4420 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:40:56.515461 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:56.515440    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:40:56.516205 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:56.516184    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:40:58.572417 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:58.572383    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:58.914692 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:58.914613    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:40:59.431264 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:40:59.431218    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:02.652430 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:41:02.652391    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:41:03.115488 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:03.115449    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:03.136571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:03.136536    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"]
Nov 04 03:41:03.265882 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:03.265826    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:03.301129 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:03.301097    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:41:03.413290 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:03.413187    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:03.897110 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:03.897073    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:04.046486 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:04.046444    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:07.925294 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:07.925261    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:08.137081 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.137038    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9" podUID="b3d2aa1f-c0c2-49a7-8b25-202c43d02133" containerName="payload-processing" containerID="cri-o://b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3" gracePeriod=25
Nov 04 03:41:08.392392 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.392364    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:41:08.480402 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.480369    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-kube-api-access-2l7xv\" (UniqueName: \"kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-kube-api-access-2l7xv\") pod \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") "
Nov 04 03:41:08.480577 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.480464    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-routing-overlay-volume\") pod \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") "
Nov 04 03:41:08.480577 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.480551    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-plugins-config-volume\") pod \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") "
Nov 04 03:41:08.480692 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.480581    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-provider-credentials\" (UniqueName: \"kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-provider-credentials\") pod \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\" (UID: \"b3d2aa1f-c0c2-49a7-8b25-202c43d02133\") "
Nov 04 03:41:08.480863 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.480819    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-routing-overlay-volume" pod "b3d2aa1f-c0c2-49a7-8b25-202c43d02133" (UID: "b3d2aa1f-c0c2-49a7-8b25-202c43d02133"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:41:08.481006 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.480939    4420 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-routing-overlay-volume\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:08.481066 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.481006    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-plugins-config-volume" pod "b3d2aa1f-c0c2-49a7-8b25-202c43d02133" (UID: "b3d2aa1f-c0c2-49a7-8b25-202c43d02133"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:41:08.482770 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.482742    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-kube-api-access-2l7xv" pod "b3d2aa1f-c0c2-49a7-8b25-202c43d02133" (UID: "b3d2aa1f-c0c2-49a7-8b25-202c43d02133"). InnerVolumeSpecName "kube-api-access-2l7xv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:41:08.482886 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.482769    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-provider-credentials" pod "b3d2aa1f-c0c2-49a7-8b25-202c43d02133" (UID: "b3d2aa1f-c0c2-49a7-8b25-202c43d02133"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:41:08.582524 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.582440    4420 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-plugins-config-volume\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:08.582524 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.582470    4420 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-provider-credentials\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:08.582524 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.582480    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l7xv\" (UniqueName: \"kubernetes.io/projected/b3d2aa1f-c0c2-49a7-8b25-202c43d02133-kube-api-access-2l7xv\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:08.814584 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:08.814550    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:09.236996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:09.236964    4420 generic.go:356] "Generic (PLEG): container finished" podID="b3d2aa1f-c0c2-49a7-8b25-202c43d02133" containerID="b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3" exitCode=0
Nov 04 03:41:09.237393 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:09.237010    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"
Nov 04 03:41:09.237393 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:09.237011    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9" event={"ID":"b3d2aa1f-c0c2-49a7-8b25-202c43d02133","Type":"ContainerDied","Data":"b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3"}
Nov 04 03:41:09.237393 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:09.237131    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9" event={"ID":"b3d2aa1f-c0c2-49a7-8b25-202c43d02133","Type":"ContainerDied","Data":"9e3fa6d7a69047696385da8b295a6877401ff59f941021094b779eb7e966e51b"}
Nov 04 03:41:09.237393 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:09.237160    4420 scope.go:122] "RemoveContainer" containerID="b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3"
Nov 04 03:41:09.247093 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:09.247076    4420 scope.go:122] "RemoveContainer" containerID="b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3"
Nov 04 03:41:09.247351 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:41:09.247333    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3\": container with ID starting with b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3 not found: ID does not exist" containerID="b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3"
Nov 04 03:41:09.247420 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:09.247358    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3"} err="failed to get container status \"b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3\": rpc error: code = NotFound desc = could not find container \"b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3\": container with ID starting with b4d298ec1d63fb2897897ef75bc9d5f0e6ca3b82fe3debda6d4b01ebf705b2e3 not found: ID does not exist"
Nov 04 03:41:09.396803 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:09.396770    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"]
Nov 04 03:41:09.443453 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:09.443416    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-1bf4bf/payload-processing-external-model-e2e-extmod-b-1bf4bf-9df8xbsl9"]
Nov 04 03:41:09.834646 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:09.834608    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:09.985582 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:09.985553    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:10.140190 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:10.140093    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:10.306226 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:10.306197    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3d2aa1f-c0c2-49a7-8b25-202c43d02133" path="/var/lib/kubelet/pods/b3d2aa1f-c0c2-49a7-8b25-202c43d02133/volumes"
Nov 04 03:41:10.978025 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:10.977946    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:41:13.301340 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:13.301306    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:41:13.726079 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:13.725993    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:14.999924 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:14.999892    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"]
Nov 04 03:41:15.000328 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.000315    4420 cpu_manager.go:414] "containerMap: removing container" podUID="7ad0f554-b218-4d84-9c72-6ca76a6361a7" containerName="revoke-keys"
Nov 04 03:41:15.000384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.000329    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ad0f554-b218-4d84-9c72-6ca76a6361a7" containerName="revoke-keys"
Nov 04 03:41:15.000384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.000340    4420 cpu_manager.go:414] "containerMap: removing container" podUID="b3d2aa1f-c0c2-49a7-8b25-202c43d02133" containerName="payload-processing"
Nov 04 03:41:15.000384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.000346    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3d2aa1f-c0c2-49a7-8b25-202c43d02133" containerName="payload-processing"
Nov 04 03:41:15.000474 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.000437    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ad0f554-b218-4d84-9c72-6ca76a6361a7" containerName="revoke-keys"
Nov 04 03:41:15.000474 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.000446    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3d2aa1f-c0c2-49a7-8b25-202c43d02133" containerName="payload-processing"
Nov 04 03:41:15.005123 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.005103    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:15.046661 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.046636    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56czg\" (UniqueName: \"kubernetes.io/projected/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-kube-api-access-56czg\") pod \"maas-api-revoke-sub-20735c2f-rf2q9\" (UID: \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:15.046796 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.046684    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-20735c2f-rf2q9\" (UID: \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:15.046894 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.046868    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-xks-ca\") pod \"maas-api-revoke-sub-20735c2f-rf2q9\" (UID: \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:15.102324 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.102301    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:41:15.102324 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.102300    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 03:41:15.102324 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.102327    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:41:15.102534 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.102301    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 03:41:15.104590 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.104564    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"]
Nov 04 03:41:15.148213 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.148186    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-xks-ca\") pod \"maas-api-revoke-sub-20735c2f-rf2q9\" (UID: \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:15.148373 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.148287    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56czg\" (UniqueName: \"kubernetes.io/projected/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-kube-api-access-56czg\") pod \"maas-api-revoke-sub-20735c2f-rf2q9\" (UID: \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:15.148373 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.148350    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-20735c2f-rf2q9\" (UID: \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:15.149004 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.148987    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-20735c2f-rf2q9\" (UID: \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:15.150529 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.150512    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-xks-ca\") pod \"maas-api-revoke-sub-20735c2f-rf2q9\" (UID: \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:15.205921 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.205889    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56czg\" (UniqueName: \"kubernetes.io/projected/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-kube-api-access-56czg\") pod \"maas-api-revoke-sub-20735c2f-rf2q9\" (UID: \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:15.317892 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.317797    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:15.690765 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:15.690679    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"]
Nov 04 03:41:15.696924 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:41:15.696893    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd489cdc1_ec1b_47f8_a09b_b25c09ee7dff.slice/crio-0a38f8a5ddc09fc526aebd417f22f9795c9b4d3eba4dde7d9b299741f5f9a2d0 WatchSource:0}: Error finding container 0a38f8a5ddc09fc526aebd417f22f9795c9b4d3eba4dde7d9b299741f5f9a2d0: Status 404 returned error can't find the container with id 0a38f8a5ddc09fc526aebd417f22f9795c9b4d3eba4dde7d9b299741f5f9a2d0
Nov 04 03:41:16.018884 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:16.017217    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:16.214196 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:16.214104    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:16.276339 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:16.276297    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9" event={"ID":"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff","Type":"ContainerStarted","Data":"990c8deefc620c1f17d1b7ac4a6b471226070b46464c0b1e3e3bef48f59fda74"}
Nov 04 03:41:16.276539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:16.276394    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9" event={"ID":"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff","Type":"ContainerStarted","Data":"0a38f8a5ddc09fc526aebd417f22f9795c9b4d3eba4dde7d9b299741f5f9a2d0"}
Nov 04 03:41:16.386859 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:16.386794    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9" podStartSLOduration=2.386779853 podStartE2EDuration="2.386779853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:41:16.385320177 +0000 UTC m=+1446.200111456" watchObservedRunningTime="2026-11-04 03:41:16.386779853 +0000 UTC m=+1446.201571130"
Nov 04 03:41:16.513479 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:16.513438    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:17.283422 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:17.283390    4420 generic.go:356] "Generic (PLEG): container finished" podID="d489cdc1-ec1b-47f8-a09b-b25c09ee7dff" containerID="990c8deefc620c1f17d1b7ac4a6b471226070b46464c0b1e3e3bef48f59fda74" exitCode=0
Nov 04 03:41:17.283820 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:17.283435    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9" event={"ID":"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff","Type":"ContainerDied","Data":"990c8deefc620c1f17d1b7ac4a6b471226070b46464c0b1e3e3bef48f59fda74"}
Nov 04 03:41:18.419204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:18.419181    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:18.488126 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:18.488096    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-kube-api-access-56czg\" (UniqueName: \"kubernetes.io/projected/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-kube-api-access-56czg\") pod \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\" (UID: \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\") "
Nov 04 03:41:18.488324 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:18.488186    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-xks-ca\") pod \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\" (UID: \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\") "
Nov 04 03:41:18.488324 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:18.488244    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-openshift-ca\") pod \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\" (UID: \"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff\") "
Nov 04 03:41:18.488640 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:18.488620    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-openshift-ca" pod "d489cdc1-ec1b-47f8-a09b-b25c09ee7dff" (UID: "d489cdc1-ec1b-47f8-a09b-b25c09ee7dff"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:41:18.490221 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:18.490192    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-xks-ca" pod "d489cdc1-ec1b-47f8-a09b-b25c09ee7dff" (UID: "d489cdc1-ec1b-47f8-a09b-b25c09ee7dff"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:41:18.490351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:18.490242    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-kube-api-access-56czg" pod "d489cdc1-ec1b-47f8-a09b-b25c09ee7dff" (UID: "d489cdc1-ec1b-47f8-a09b-b25c09ee7dff"). InnerVolumeSpecName "kube-api-access-56czg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:41:18.589650 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:18.589570    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56czg\" (UniqueName: \"kubernetes.io/projected/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-kube-api-access-56czg\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:18.589650 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:18.589598    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-xks-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:18.589650 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:18.589609    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff-maas-api-openshift-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:41:18.594939 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:18.594906    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:19.295567 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:19.295529    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9" event={"ID":"d489cdc1-ec1b-47f8-a09b-b25c09ee7dff","Type":"ContainerDied","Data":"0a38f8a5ddc09fc526aebd417f22f9795c9b4d3eba4dde7d9b299741f5f9a2d0"}
Nov 04 03:41:19.295721 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:19.295576    4420 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a38f8a5ddc09fc526aebd417f22f9795c9b4d3eba4dde7d9b299741f5f9a2d0"
Nov 04 03:41:19.295721 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:19.295543    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"
Nov 04 03:41:22.565410 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:22.565370    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:23.301080 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:23.301046    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:41:23.531918 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:23.531875    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:28.353814 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:28.353782    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:28.621446 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:28.621351    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:28.797396 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:28.797349    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:28.968444 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:28.968347    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:29.194270 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:29.194227    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:32.650746 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:41:32.650705    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:41:33.301864 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:33.301801    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:41:33.587615 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:33.587528    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:33.767349 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:33.767315    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:37.171931 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:37.171890    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:38.001887 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:38.001829    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:38.936607 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:38.936566    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:39.089190 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:39.089154    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:39.243357 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:39.243321    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:39.515619 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:39.515536    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:39.679084 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:39.679046    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:39.827961 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:39.827877    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:39.986687 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:39.986651    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:40.531718 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:40.531675    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:40.736874 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:40.736819    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:42.933124 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:42.933078    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:43.133437 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:43.133404    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:43.301286 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:43.301244    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:41:44.520305 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:44.520262    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:44.693214 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:44.693182    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:45.052257 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:45.052223    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:45.217196 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:45.217162    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:45.882503 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:45.882466    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:46.034608 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:46.034576    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:46.183301 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:46.183215    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:47.048836 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:47.048802    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:47.258910 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:47.258875    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:50.306861 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:50.306825    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:52.148966 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:52.148887    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:52.346890 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:52.346854    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:52.583438 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:52.583405    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:52.732023 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:52.731979    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:53.171283 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:53.171248    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:53.301415 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:53.301371    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:41:54.896196 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:54.896163    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:41:55.379154 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:55.379123    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:58.740202 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:58.740168    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:58.934020 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:58.933980    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:41:59.476977 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:41:59.476937    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:00.142318 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:42:00.142270    4420 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-243e19a17a73: secret "agc-cred-243e19a17a73" not found
Nov 04 03:42:00.142318 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:42:00.142306    4420 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d: secret "agc-cred-243e19a17a73" not found
Nov 04 03:42:00.142804 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:42:00.142388    4420 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials podName:8165ded7-cbdd-41e6-9790-06ad19ab9a2e nodeName:}" failed. No retries permitted until 2026-11-04 03:44:02.142366092 +0000 UTC m=+1611.957157371 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials") pod "payload-processing-external-model-7f5c68695b-7z22d" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e") : secret "agc-cred-243e19a17a73" not found
Nov 04 03:42:00.542425 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:00.542385    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:02.649496 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:42:02.649457    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:42:03.301122 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:03.301089    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:42:05.500795 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:05.500760    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:05.707076 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:05.707036    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:08.271033 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:08.270992    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:10.324227 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:10.324191    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:10.458045 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:10.457906    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:42:10.458272 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:10.458251    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:42:13.301427 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:13.301390    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:42:15.169293 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:15.169260    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:20.123770 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:20.123726    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:23.301806 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:23.301764    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:42:25.043950 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:25.043914    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:29.910003 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:29.909962    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:31.075265 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:31.075231    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:42:32.649408 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:42:32.649369    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:42:33.301299 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.301266    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:42:33.538133 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.538099    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"]
Nov 04 03:42:33.538608 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.538594    4420 cpu_manager.go:414] "containerMap: removing container" podUID="d489cdc1-ec1b-47f8-a09b-b25c09ee7dff" containerName="revoke-keys"
Nov 04 03:42:33.538652 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.538610    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d489cdc1-ec1b-47f8-a09b-b25c09ee7dff" containerName="revoke-keys"
Nov 04 03:42:33.538686 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.538679    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="d489cdc1-ec1b-47f8-a09b-b25c09ee7dff" containerName="revoke-keys"
Nov 04 03:42:33.541941 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.541921    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:33.591174 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.591106    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 03:42:33.637726 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.637693    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 03:42:33.637930 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.637762    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:42:33.637930 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.637822    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 03:42:33.640287 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.640261    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"]
Nov 04 03:42:33.712022 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.711986    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5nfv\" (UniqueName: \"kubernetes.io/projected/301f8896-55ce-4194-8c92-89c352d19a4c-kube-api-access-s5nfv\") pod \"maas-api-revoke-sub-53dc6c4c-zf96d\" (UID: \"301f8896-55ce-4194-8c92-89c352d19a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:33.712478 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.712052    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-53dc6c4c-zf96d\" (UID: \"301f8896-55ce-4194-8c92-89c352d19a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:33.712478 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.712219    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-53dc6c4c-zf96d\" (UID: \"301f8896-55ce-4194-8c92-89c352d19a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:33.813657 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.813629    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-53dc6c4c-zf96d\" (UID: \"301f8896-55ce-4194-8c92-89c352d19a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:33.813864 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.813688    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5nfv\" (UniqueName: \"kubernetes.io/projected/301f8896-55ce-4194-8c92-89c352d19a4c-kube-api-access-s5nfv\") pod \"maas-api-revoke-sub-53dc6c4c-zf96d\" (UID: \"301f8896-55ce-4194-8c92-89c352d19a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:33.820587 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.813725    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-53dc6c4c-zf96d\" (UID: \"301f8896-55ce-4194-8c92-89c352d19a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:33.820587 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.814791    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-53dc6c4c-zf96d\" (UID: \"301f8896-55ce-4194-8c92-89c352d19a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:33.820587 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.816972    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-53dc6c4c-zf96d\" (UID: \"301f8896-55ce-4194-8c92-89c352d19a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:33.871505 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:33.871441    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5nfv\" (UniqueName: \"kubernetes.io/projected/301f8896-55ce-4194-8c92-89c352d19a4c-kube-api-access-s5nfv\") pod \"maas-api-revoke-sub-53dc6c4c-zf96d\" (UID: \"301f8896-55ce-4194-8c92-89c352d19a4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:34.154527 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:34.154437    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:34.167256 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:34.167229    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:34.532438 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:34.532342    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"]
Nov 04 03:42:34.537401 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:42:34.537372    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod301f8896_55ce_4194_8c92_89c352d19a4c.slice/crio-586f070d0d16a34a48e4d482b37252ea01f1ff223a15e26a06a1a0bae0474678 WatchSource:0}: Error finding container 586f070d0d16a34a48e4d482b37252ea01f1ff223a15e26a06a1a0bae0474678: Status 404 returned error can't find the container with id 586f070d0d16a34a48e4d482b37252ea01f1ff223a15e26a06a1a0bae0474678
Nov 04 03:42:34.694405 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:34.694370    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d" event={"ID":"301f8896-55ce-4194-8c92-89c352d19a4c","Type":"ContainerStarted","Data":"586f070d0d16a34a48e4d482b37252ea01f1ff223a15e26a06a1a0bae0474678"}
Nov 04 03:42:35.342334 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:35.342302    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:35.702139 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:35.702047    4420 generic.go:356] "Generic (PLEG): container finished" podID="301f8896-55ce-4194-8c92-89c352d19a4c" containerID="97691b32ac623d9f76091797aaf27de46d25962c0774ec31511930d82738cea6" exitCode=0
Nov 04 03:42:35.702139 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:35.702123    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d" event={"ID":"301f8896-55ce-4194-8c92-89c352d19a4c","Type":"ContainerDied","Data":"97691b32ac623d9f76091797aaf27de46d25962c0774ec31511930d82738cea6"}
Nov 04 03:42:36.845523 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:36.845501    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:36.950772 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:36.950733    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/301f8896-55ce-4194-8c92-89c352d19a4c-kube-api-access-s5nfv\" (UniqueName: \"kubernetes.io/projected/301f8896-55ce-4194-8c92-89c352d19a4c-kube-api-access-s5nfv\") pod \"301f8896-55ce-4194-8c92-89c352d19a4c\" (UID: \"301f8896-55ce-4194-8c92-89c352d19a4c\") "
Nov 04 03:42:36.950980 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:36.950792    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-openshift-ca\") pod \"301f8896-55ce-4194-8c92-89c352d19a4c\" (UID: \"301f8896-55ce-4194-8c92-89c352d19a4c\") "
Nov 04 03:42:36.950980 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:36.950938    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-xks-ca\") pod \"301f8896-55ce-4194-8c92-89c352d19a4c\" (UID: \"301f8896-55ce-4194-8c92-89c352d19a4c\") "
Nov 04 03:42:36.951224 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:36.951199    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-openshift-ca" pod "301f8896-55ce-4194-8c92-89c352d19a4c" (UID: "301f8896-55ce-4194-8c92-89c352d19a4c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:36.952914 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:36.952884    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-xks-ca" pod "301f8896-55ce-4194-8c92-89c352d19a4c" (UID: "301f8896-55ce-4194-8c92-89c352d19a4c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:42:36.953015 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:36.952949    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/301f8896-55ce-4194-8c92-89c352d19a4c-kube-api-access-s5nfv" pod "301f8896-55ce-4194-8c92-89c352d19a4c" (UID: "301f8896-55ce-4194-8c92-89c352d19a4c"). InnerVolumeSpecName "kube-api-access-s5nfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:37.052560 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:37.052530    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-xks-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:37.052560 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:37.052559    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5nfv\" (UniqueName: \"kubernetes.io/projected/301f8896-55ce-4194-8c92-89c352d19a4c-kube-api-access-s5nfv\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:37.052560 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:37.052568    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/301f8896-55ce-4194-8c92-89c352d19a4c-maas-api-openshift-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:37.117421 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:37.117384    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:37.714384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:37.714358    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"
Nov 04 03:42:37.714384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:37.714380    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d" event={"ID":"301f8896-55ce-4194-8c92-89c352d19a4c","Type":"ContainerDied","Data":"586f070d0d16a34a48e4d482b37252ea01f1ff223a15e26a06a1a0bae0474678"}
Nov 04 03:42:37.714651 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:37.714422    4420 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="586f070d0d16a34a48e4d482b37252ea01f1ff223a15e26a06a1a0bae0474678"
Nov 04 03:42:37.760453 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:37.760415    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:37.913541 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:37.913506    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:38.731669 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:38.731633    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"]
Nov 04 03:42:38.731890 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:38.731838    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" podUID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" containerName="istio-proxy" containerID="cri-o://3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342" gracePeriod=30
Nov 04 03:42:39.305956 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:39.305917    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:39.796770 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:39.796731    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:40.559731 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:40.559692    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:40.761578 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:40.761546    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:41.193037 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:41.193003    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:41.647460 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:41.647424    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:42.423640 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:42.423609    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:43.301203 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:43.301163    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:42:43.991097 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:43.991074    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:42:44.025415 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025335    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-data\") pod \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") "
Nov 04 03:42:44.025415 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025390    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-envoy\") pod \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") "
Nov 04 03:42:44.025637 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025436    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istiod-ca-cert\") pod \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") "
Nov 04 03:42:44.025637 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025457    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-certs\") pod \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") "
Nov 04 03:42:44.025637 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025484    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-token\" (UniqueName: \"kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-token\") pod \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") "
Nov 04 03:42:44.025637 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025514    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-podinfo\") pod \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") "
Nov 04 03:42:44.025637 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025569    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-socket\") pod \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") "
Nov 04 03:42:44.025940 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025649    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-kube-api-access-qgrhs\" (UniqueName: \"kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-kube-api-access-qgrhs\") pod \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") "
Nov 04 03:42:44.025940 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025662    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-data" pod "6c75e12d-2787-43af-a6f2-1e678b9c33f3" (UID: "6c75e12d-2787-43af-a6f2-1e678b9c33f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:44.025940 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025802    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-credential-socket\") pod \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\" (UID: \"6c75e12d-2787-43af-a6f2-1e678b9c33f3\") "
Nov 04 03:42:44.025940 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025804    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-certs" pod "6c75e12d-2787-43af-a6f2-1e678b9c33f3" (UID: "6c75e12d-2787-43af-a6f2-1e678b9c33f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:44.025940 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025912    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istiod-ca-cert" pod "6c75e12d-2787-43af-a6f2-1e678b9c33f3" (UID: "6c75e12d-2787-43af-a6f2-1e678b9c33f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:42:44.026178 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.025955    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-socket" pod "6c75e12d-2787-43af-a6f2-1e678b9c33f3" (UID: "6c75e12d-2787-43af-a6f2-1e678b9c33f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:44.026178 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.026062    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-credential-socket" pod "6c75e12d-2787-43af-a6f2-1e678b9c33f3" (UID: "6c75e12d-2787-43af-a6f2-1e678b9c33f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:44.026339 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.026319    4420 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istiod-ca-cert\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:44.026407 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.026347    4420 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:44.026407 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.026362    4420 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-workload-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:44.026407 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.026377    4420 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-credential-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:44.026407 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.026392    4420 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-data\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:44.027698 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.027664    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-token" pod "6c75e12d-2787-43af-a6f2-1e678b9c33f3" (UID: "6c75e12d-2787-43af-a6f2-1e678b9c33f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:44.027802 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.027744    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-kube-api-access-qgrhs" pod "6c75e12d-2787-43af-a6f2-1e678b9c33f3" (UID: "6c75e12d-2787-43af-a6f2-1e678b9c33f3"). InnerVolumeSpecName "kube-api-access-qgrhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:42:44.027802 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.027757    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-podinfo" pod "6c75e12d-2787-43af-a6f2-1e678b9c33f3" (UID: "6c75e12d-2787-43af-a6f2-1e678b9c33f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:42:44.028091 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.028066    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-envoy" pod "6c75e12d-2787-43af-a6f2-1e678b9c33f3" (UID: "6c75e12d-2787-43af-a6f2-1e678b9c33f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:42:44.126943 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.126907    4420 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-envoy\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:44.126943 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.126936    4420 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-token\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:44.126943 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.126944    4420 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c75e12d-2787-43af-a6f2-1e678b9c33f3-istio-podinfo\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:44.127180 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.126955    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgrhs\" (UniqueName: \"kubernetes.io/projected/6c75e12d-2787-43af-a6f2-1e678b9c33f3-kube-api-access-qgrhs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:42:44.757123 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.757093    4420 generic.go:356] "Generic (PLEG): container finished" podID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" containerID="3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342" exitCode=0
Nov 04 03:42:44.757545 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.757152    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"
Nov 04 03:42:44.757545 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.757172    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" event={"ID":"6c75e12d-2787-43af-a6f2-1e678b9c33f3","Type":"ContainerDied","Data":"3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342"}
Nov 04 03:42:44.757545 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.757229    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd" event={"ID":"6c75e12d-2787-43af-a6f2-1e678b9c33f3","Type":"ContainerDied","Data":"36c7c8c3eabe8e4a1fd8ffb75c35e8604edc5a1f8f1e1c9ab746d7f61ac998c2"}
Nov 04 03:42:44.757545 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.757252    4420 scope.go:122] "RemoveContainer" containerID="3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342"
Nov 04 03:42:44.767022 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.767001    4420 scope.go:122] "RemoveContainer" containerID="3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342"
Nov 04 03:42:44.767304 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:42:44.767280    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342\": container with ID starting with 3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342 not found: ID does not exist" containerID="3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342"
Nov 04 03:42:44.767380 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.767314    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342"} err="failed to get container status \"3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342\": rpc error: code = NotFound desc = could not find container \"3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342\": container with ID starting with 3ef030a7c9dbec7f51e12be1c91fb96425fbfb0efd669c5289a888447d8e8342 not found: ID does not exist"
Nov 04 03:42:44.974704 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:44.974668    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"]
Nov 04 03:42:45.021226 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:45.021145    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-fba151-openshift-default-d5688d5cc-vkhpd"]
Nov 04 03:42:46.306634 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:46.306597    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" path="/var/lib/kubelet/pods/6c75e12d-2787-43af-a6f2-1e678b9c33f3/volumes"
Nov 04 03:42:46.720795 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:46.720708    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:46.923489 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:46.923453    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:47.287994 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:47.287959    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:47.441621 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:47.441575    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:47.693456 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:47.693361    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:47.837052 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:47.837002    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:47.988675 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:47.988646    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:51.870255 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:51.870218    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:52.832713 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:52.832674    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:53.302099 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:53.302064    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:42:54.355340 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:54.355288    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:54.508396 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:54.508356    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:54.655804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:54.655725    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:54.856792 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:54.856702    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:55.075898 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:55.075827    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:57.428306 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:57.428273    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:42:59.410725 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:42:59.410686    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:00.796301 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:00.796267    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:00.943032 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:00.943001    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:01.180231 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:01.180146    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:01.326986 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:01.326945    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:02.362713 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:02.362678    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:02.527263 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:02.527221    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:02.650954 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:43:02.650827    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:43:02.681591 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:02.681555    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:02.887815 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:02.887785    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:03.301630 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:03.301585    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:43:03.378729 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:03.378684    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:04.798106 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:04.798075    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:06.658104 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:06.658066    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:43:07.823162 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:07.823121    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:08.333649 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:08.333595    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"]
Nov 04 03:43:08.526679 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:08.526646    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:08.690391 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:08.690302    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:09.356762 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:09.356728    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"]
Nov 04 03:43:09.359338 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:09.359288    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 03:43:10.303864 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:10.300766    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:12.661926 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:12.661889    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:12.832081 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:12.832041    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:14.358002 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.357952    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing" containerID="cri-o://a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a" gracePeriod=25
Nov 04 03:43:14.616317 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.616257    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:43:14.679443 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.679415    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-provider-credentials\" (UniqueName: \"kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-provider-credentials\") pod \"156e2961-9ef5-43b5-8378-893caecef3d4\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") "
Nov 04 03:43:14.679588 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.679459    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-plugins-config-volume\") pod \"156e2961-9ef5-43b5-8378-893caecef3d4\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") "
Nov 04 03:43:14.679588 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.679550    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-kube-api-access-bnthb\" (UniqueName: \"kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-kube-api-access-bnthb\") pod \"156e2961-9ef5-43b5-8378-893caecef3d4\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") "
Nov 04 03:43:14.679672 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.679585    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-routing-overlay-volume\") pod \"156e2961-9ef5-43b5-8378-893caecef3d4\" (UID: \"156e2961-9ef5-43b5-8378-893caecef3d4\") "
Nov 04 03:43:14.680013 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.679982    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-plugins-config-volume" pod "156e2961-9ef5-43b5-8378-893caecef3d4" (UID: "156e2961-9ef5-43b5-8378-893caecef3d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:14.680013 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.679999    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-routing-overlay-volume" pod "156e2961-9ef5-43b5-8378-893caecef3d4" (UID: "156e2961-9ef5-43b5-8378-893caecef3d4"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:14.681594 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.681575    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-kube-api-access-bnthb" pod "156e2961-9ef5-43b5-8378-893caecef3d4" (UID: "156e2961-9ef5-43b5-8378-893caecef3d4"). InnerVolumeSpecName "kube-api-access-bnthb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:14.681737 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.681689    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-provider-credentials" pod "156e2961-9ef5-43b5-8378-893caecef3d4" (UID: "156e2961-9ef5-43b5-8378-893caecef3d4"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:14.780768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.780734    4420 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-plugins-config-volume\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:14.780768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.780762    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnthb\" (UniqueName: \"kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-kube-api-access-bnthb\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:14.780768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.780771    4420 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/156e2961-9ef5-43b5-8378-893caecef3d4-routing-overlay-volume\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:14.781039 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.780779    4420 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/156e2961-9ef5-43b5-8378-893caecef3d4-provider-credentials\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:14.958498 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.958406    4420 generic.go:356] "Generic (PLEG): container finished" podID="156e2961-9ef5-43b5-8378-893caecef3d4" containerID="a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a" exitCode=0
Nov 04 03:43:14.958498 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.958449    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"
Nov 04 03:43:14.958678 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.958498    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" event={"ID":"156e2961-9ef5-43b5-8378-893caecef3d4","Type":"ContainerDied","Data":"a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a"}
Nov 04 03:43:14.958678 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.958553    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz" event={"ID":"156e2961-9ef5-43b5-8378-893caecef3d4","Type":"ContainerDied","Data":"baba4e0f82f57d0e77b942be4c4ad2a4cc52345abf3fb873f4b49b2f65ee9d67"}
Nov 04 03:43:14.958678 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.958576    4420 scope.go:122] "RemoveContainer" containerID="a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a"
Nov 04 03:43:14.967571 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.967551    4420 scope.go:122] "RemoveContainer" containerID="a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a"
Nov 04 03:43:14.967831 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:43:14.967811    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a\": container with ID starting with a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a not found: ID does not exist" containerID="a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a"
Nov 04 03:43:14.967927 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:14.967838    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a"} err="failed to get container status \"a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a\": rpc error: code = NotFound desc = could not find container \"a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a\": container with ID starting with a335b33fc375d70caba81c1500c249505fd7ae1a6b9c77ad18987938e0d8d66a not found: ID does not exist"
Nov 04 03:43:15.116078 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:15.116038    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"]
Nov 04 03:43:15.163133 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:15.163100    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-8b89f9dc6-zbkqz"]
Nov 04 03:43:15.230021 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:15.229932    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:15.445555 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:15.445518    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:16.306467 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:16.306431    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" path="/var/lib/kubelet/pods/156e2961-9ef5-43b5-8378-893caecef3d4/volumes"
Nov 04 03:43:20.214661 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:20.214626    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:21.735513 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:21.735420    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:21.886688 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:21.886633    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:22.044162 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:22.044127    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:22.169839 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:22.169805    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"]
Nov 04 03:43:22.945564 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:22.945524    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:23.108727 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.108688    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:23.355270 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.355231    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb" podUID="85dd0445-88e0-4c88-8d9a-14fbf6bc2656" containerName="main" containerID="cri-o://8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d" gracePeriod=45
Nov 04 03:43:23.622243 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.622218    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:43:23.678487 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.678458    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-dshm\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-dshm\") pod \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") "
Nov 04 03:43:23.678657 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.678527    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-home\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-home\") pod \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") "
Nov 04 03:43:23.678657 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.678551    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tmp-dir\") pod \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") "
Nov 04 03:43:23.678657 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.678589    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tls-certs\" (UniqueName: \"kubernetes.io/secret/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tls-certs\") pod \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") "
Nov 04 03:43:23.678657 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.678615    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-kube-api-access-fhh6x\" (UniqueName: \"kubernetes.io/projected/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-kube-api-access-fhh6x\") pod \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") "
Nov 04 03:43:23.678657 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.678641    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-model-cache\") pod \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\" (UID: \"85dd0445-88e0-4c88-8d9a-14fbf6bc2656\") "
Nov 04 03:43:23.678965 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.678788    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-home" pod "85dd0445-88e0-4c88-8d9a-14fbf6bc2656" (UID: "85dd0445-88e0-4c88-8d9a-14fbf6bc2656"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:23.678965 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.678881    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tmp-dir" pod "85dd0445-88e0-4c88-8d9a-14fbf6bc2656" (UID: "85dd0445-88e0-4c88-8d9a-14fbf6bc2656"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:23.679115 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.679092    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-model-cache" pod "85dd0445-88e0-4c88-8d9a-14fbf6bc2656" (UID: "85dd0445-88e0-4c88-8d9a-14fbf6bc2656"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:23.679234 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.679114    4420 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-home\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:23.679234 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.679134    4420 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tmp-dir\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:23.680898 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.680874    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tls-certs" pod "85dd0445-88e0-4c88-8d9a-14fbf6bc2656" (UID: "85dd0445-88e0-4c88-8d9a-14fbf6bc2656"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:43:23.680995 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.680911    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-dshm" pod "85dd0445-88e0-4c88-8d9a-14fbf6bc2656" (UID: "85dd0445-88e0-4c88-8d9a-14fbf6bc2656"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:43:23.681034 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.680990    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-kube-api-access-fhh6x" pod "85dd0445-88e0-4c88-8d9a-14fbf6bc2656" (UID: "85dd0445-88e0-4c88-8d9a-14fbf6bc2656"). InnerVolumeSpecName "kube-api-access-fhh6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:23.780405 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.780373    4420 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-dshm\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:23.780405 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.780410    4420 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-tls-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:23.780604 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.780425    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhh6x\" (UniqueName: \"kubernetes.io/projected/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-kube-api-access-fhh6x\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:23.780604 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:23.780443    4420 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85dd0445-88e0-4c88-8d9a-14fbf6bc2656-model-cache\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:24.013659 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:24.013622    4420 generic.go:356] "Generic (PLEG): container finished" podID="85dd0445-88e0-4c88-8d9a-14fbf6bc2656" containerID="8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d" exitCode=0
Nov 04 03:43:24.014107 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:24.013685    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb" event={"ID":"85dd0445-88e0-4c88-8d9a-14fbf6bc2656","Type":"ContainerDied","Data":"8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d"}
Nov 04 03:43:24.014107 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:24.013696    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"
Nov 04 03:43:24.014107 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:24.013718    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb" event={"ID":"85dd0445-88e0-4c88-8d9a-14fbf6bc2656","Type":"ContainerDied","Data":"01c48d1576a0ce815d4559b517d989b537dc42f541b6945f844921b2c48a6b9c"}
Nov 04 03:43:24.014107 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:24.013735    4420 scope.go:122] "RemoveContainer" containerID="8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d"
Nov 04 03:43:24.023671 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:24.023653    4420 scope.go:122] "RemoveContainer" containerID="8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d"
Nov 04 03:43:24.024012 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:43:24.023995    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d\": container with ID starting with 8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d not found: ID does not exist" containerID="8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d"
Nov 04 03:43:24.024056 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:24.024022    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d"} err="failed to get container status \"8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d\": rpc error: code = NotFound desc = could not find container \"8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d\": container with ID starting with 8e770a6b83855d0b1928881e1ab9a5d41ea093519ea146c2063a8674d5cc9c6d not found: ID does not exist"
Nov 04 03:43:24.173172 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:24.173136    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"]
Nov 04 03:43:24.177490 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:24.177458    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-82c724/simulator-w3-82c724-kserve-6fcd6fccdd-zjkqb"]
Nov 04 03:43:24.306537 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:24.306467    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85dd0445-88e0-4c88-8d9a-14fbf6bc2656" path="/var/lib/kubelet/pods/85dd0445-88e0-4c88-8d9a-14fbf6bc2656/volumes"
Nov 04 03:43:24.744409 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:24.744380    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:25.103001 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:25.102905    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:25.257056 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:25.257014    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:25.654233 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:25.654143    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:26.458520 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:26.458478    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:27.171146 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.171108    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d" podUID="8165ded7-cbdd-41e6-9790-06ad19ab9a2e" containerName="payload-processing" containerID="cri-o://fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea" gracePeriod=25
Nov 04 03:43:27.386494 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.386464    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:27.437003 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.436944    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:43:27.521288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.521253    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials\" (UniqueName: \"kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials\") pod \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") "
Nov 04 03:43:27.521762 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.521299    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-kube-api-access-xtkqd\" (UniqueName: \"kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-kube-api-access-xtkqd\") pod \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") "
Nov 04 03:43:27.521762 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.521322    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-plugins-config-volume\") pod \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") "
Nov 04 03:43:27.521762 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.521358    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-routing-overlay-volume\") pod \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\" (UID: \"8165ded7-cbdd-41e6-9790-06ad19ab9a2e\") "
Nov 04 03:43:27.521762 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.521707    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-routing-overlay-volume" pod "8165ded7-cbdd-41e6-9790-06ad19ab9a2e" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:27.521762 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.521741    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-plugins-config-volume" pod "8165ded7-cbdd-41e6-9790-06ad19ab9a2e" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:43:27.523361 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.523333    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-kube-api-access-xtkqd" pod "8165ded7-cbdd-41e6-9790-06ad19ab9a2e" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e"). InnerVolumeSpecName "kube-api-access-xtkqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:27.523486 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.523454    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials" pod "8165ded7-cbdd-41e6-9790-06ad19ab9a2e" (UID: "8165ded7-cbdd-41e6-9790-06ad19ab9a2e"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:43:27.623175 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.623142    4420 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-provider-credentials\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:27.623175 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.623172    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtkqd\" (UniqueName: \"kubernetes.io/projected/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-kube-api-access-xtkqd\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:27.623175 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.623183    4420 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-plugins-config-volume\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:27.623413 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:27.623192    4420 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8165ded7-cbdd-41e6-9790-06ad19ab9a2e-routing-overlay-volume\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:43:28.037068 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:28.037034    4420 generic.go:356] "Generic (PLEG): container finished" podID="8165ded7-cbdd-41e6-9790-06ad19ab9a2e" containerID="fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea" exitCode=0
Nov 04 03:43:28.037242 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:28.037075    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"
Nov 04 03:43:28.037242 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:28.037094    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d" event={"ID":"8165ded7-cbdd-41e6-9790-06ad19ab9a2e","Type":"ContainerDied","Data":"fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea"}
Nov 04 03:43:28.037242 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:28.037139    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d" event={"ID":"8165ded7-cbdd-41e6-9790-06ad19ab9a2e","Type":"ContainerDied","Data":"4322c689b52e4e4a4c5a95a085c04739a489d2c666e1b7aad98be3ea50cafdb9"}
Nov 04 03:43:28.037242 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:28.037161    4420 scope.go:122] "RemoveContainer" containerID="fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea"
Nov 04 03:43:28.046464 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:28.046438    4420 scope.go:122] "RemoveContainer" containerID="fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea"
Nov 04 03:43:28.046789 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:43:28.046767    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea\": container with ID starting with fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea not found: ID does not exist" containerID="fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea"
Nov 04 03:43:28.046833 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:28.046800    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea"} err="failed to get container status \"fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea\": rpc error: code = NotFound desc = could not find container \"fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea\": container with ID starting with fd5545273d9fbb219162637f63bb082192bd825b11cf8850e81e7f554340ccea not found: ID does not exist"
Nov 04 03:43:28.120410 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:28.120373    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:28.195864 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:28.195737    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"]
Nov 04 03:43:28.242595 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:28.242560    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-7f5c68695b-7z22d"]
Nov 04 03:43:28.306465 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:28.306396    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8165ded7-cbdd-41e6-9790-06ad19ab9a2e" path="/var/lib/kubelet/pods/8165ded7-cbdd-41e6-9790-06ad19ab9a2e/volumes"
Nov 04 03:43:29.948997 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:29.948958    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:30.836359 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:30.836255    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:43:31.022221 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:31.022169    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:31.203775 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:31.203681    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:32.023930 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:32.023894    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:32.656065 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:43:32.656026    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:43:34.390048 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:34.389999    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:34.540323 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:34.540293    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:34.936045 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:34.936011    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:37.458564 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:37.458527    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:42.563216 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:42.563178    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:45.732002 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:45.731966    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:45.912603 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:45.912564    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:46.068756 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:46.068720    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:46.581287 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:46.581250    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:46.842587 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:46.842487    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:47.755188 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:47.755145    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:52.193076 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:52.193037    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:53.942870 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:53.942603    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:54.110882 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:54.110382    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:58.834625 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:58.834587    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:58.991793 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:58.991739    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:59.240427 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:59.240389    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:43:59.785445 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:43:59.785418    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:00.111880 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:00.111763    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:00.815083 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:00.815051    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:01.424876 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:01.424822    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:01.935083 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:01.935044    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:02.652154 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:44:02.652121    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:44:04.657921 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:04.657882    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:05.763657 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:05.763624    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"]
Nov 04 03:44:05.809508 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:05.809475    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"]
Nov 04 03:44:05.811437 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:05.811415    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"]
Nov 04 03:44:05.816756 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:05.816716    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"]
Nov 04 03:44:05.962707 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:05.962677    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:07.920336 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:07.920301    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:44:09.053631 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:09.053576    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:09.821573 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:09.821480    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:10.050979 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:10.050943    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:12.588585 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:12.588550    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:15.014983 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:15.014942    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:17.016434 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:17.016387    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:17.056012 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:17.055979    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"]
Nov 04 03:44:17.102602 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:17.102566    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f6219028-c4clw"]
Nov 04 03:44:18.307035 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:18.306995    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2628336f-46ef-49fe-bc60-0d0a4db688cf" path="/var/lib/kubelet/pods/2628336f-46ef-49fe-bc60-0d0a4db688cf/volumes"
Nov 04 03:44:18.732526 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:18.732439    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:19.957424 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:19.957393    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:20.154619 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:20.154572    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:20.783858 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:20.783813    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5" podUID="3b12d15f-129b-4121-85ac-c83c3581fcc2" containerName="main" containerID="cri-o://fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3" gracePeriod=45
Nov 04 03:44:20.829538 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:20.829493    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x" podUID="f6339e78-6825-4246-a5d2-11831cd454af" containerName="main" containerID="cri-o://46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e" gracePeriod=45
Nov 04 03:44:20.830572 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:20.830544    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg" podUID="5163cb68-1d76-43af-8654-990a89eb7405" containerName="main" containerID="cri-o://50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277" gracePeriod=45
Nov 04 03:44:20.835173 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:20.835148    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt" podUID="1152bcd4-4f6e-4068-827d-02514564797d" containerName="main" containerID="cri-o://d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018" gracePeriod=45
Nov 04 03:44:21.142088 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.142062    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:44:21.145384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.145353    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:44:21.164751 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.164727    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:44:21.174712 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.174690    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:44:21.226060 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226025    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5163cb68-1d76-43af-8654-990a89eb7405-tls-certs\" (UniqueName: \"kubernetes.io/secret/5163cb68-1d76-43af-8654-990a89eb7405-tls-certs\") pod \"5163cb68-1d76-43af-8654-990a89eb7405\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") "
Nov 04 03:44:21.226263 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226078    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1152bcd4-4f6e-4068-827d-02514564797d-kube-api-access-m276p\" (UniqueName: \"kubernetes.io/projected/1152bcd4-4f6e-4068-827d-02514564797d-kube-api-access-m276p\") pod \"1152bcd4-4f6e-4068-827d-02514564797d\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") "
Nov 04 03:44:21.226263 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226110    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3b12d15f-129b-4121-85ac-c83c3581fcc2-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b12d15f-129b-4121-85ac-c83c3581fcc2-tls-certs\") pod \"3b12d15f-129b-4121-85ac-c83c3581fcc2\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") "
Nov 04 03:44:21.226263 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226147    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6339e78-6825-4246-a5d2-11831cd454af-tls-certs\" (UniqueName: \"kubernetes.io/secret/f6339e78-6825-4246-a5d2-11831cd454af-tls-certs\") pod \"f6339e78-6825-4246-a5d2-11831cd454af\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") "
Nov 04 03:44:21.226263 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226178    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-model-cache\") pod \"3b12d15f-129b-4121-85ac-c83c3581fcc2\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") "
Nov 04 03:44:21.226263 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226211    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-tmp-dir\") pod \"1152bcd4-4f6e-4068-827d-02514564797d\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") "
Nov 04 03:44:21.226263 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226246    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-home\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-home\") pod \"5163cb68-1d76-43af-8654-990a89eb7405\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") "
Nov 04 03:44:21.226522 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226300    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5163cb68-1d76-43af-8654-990a89eb7405-kube-api-access-6j5h5\" (UniqueName: \"kubernetes.io/projected/5163cb68-1d76-43af-8654-990a89eb7405-kube-api-access-6j5h5\") pod \"5163cb68-1d76-43af-8654-990a89eb7405\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") "
Nov 04 03:44:21.226522 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226327    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-tmp-dir\") pod \"f6339e78-6825-4246-a5d2-11831cd454af\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") "
Nov 04 03:44:21.226522 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226361    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b12d15f-129b-4121-85ac-c83c3581fcc2-kube-api-access-mqgwf\" (UniqueName: \"kubernetes.io/projected/3b12d15f-129b-4121-85ac-c83c3581fcc2-kube-api-access-mqgwf\") pod \"3b12d15f-129b-4121-85ac-c83c3581fcc2\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") "
Nov 04 03:44:21.226522 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226418    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-model-cache\") pod \"5163cb68-1d76-43af-8654-990a89eb7405\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") "
Nov 04 03:44:21.226522 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226486    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-home\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-home\") pod \"1152bcd4-4f6e-4068-827d-02514564797d\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") "
Nov 04 03:44:21.226774 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226538    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6339e78-6825-4246-a5d2-11831cd454af-kube-api-access-wbdq7\" (UniqueName: \"kubernetes.io/projected/f6339e78-6825-4246-a5d2-11831cd454af-kube-api-access-wbdq7\") pod \"f6339e78-6825-4246-a5d2-11831cd454af\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") "
Nov 04 03:44:21.226774 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226583    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1152bcd4-4f6e-4068-827d-02514564797d-tls-certs\" (UniqueName: \"kubernetes.io/secret/1152bcd4-4f6e-4068-827d-02514564797d-tls-certs\") pod \"1152bcd4-4f6e-4068-827d-02514564797d\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") "
Nov 04 03:44:21.226774 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226627    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-dshm\") pod \"3b12d15f-129b-4121-85ac-c83c3581fcc2\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") "
Nov 04 03:44:21.226774 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226663    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-dshm\") pod \"5163cb68-1d76-43af-8654-990a89eb7405\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") "
Nov 04 03:44:21.226774 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226718    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-tmp-dir\") pod \"5163cb68-1d76-43af-8654-990a89eb7405\" (UID: \"5163cb68-1d76-43af-8654-990a89eb7405\") "
Nov 04 03:44:21.226774 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226750    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-home\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-home\") pod \"f6339e78-6825-4246-a5d2-11831cd454af\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") "
Nov 04 03:44:21.226774 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226773    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-dshm\") pod \"1152bcd4-4f6e-4068-827d-02514564797d\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") "
Nov 04 03:44:21.227157 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226800    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-home\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-home\") pod \"3b12d15f-129b-4121-85ac-c83c3581fcc2\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") "
Nov 04 03:44:21.227157 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226832    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-tmp-dir\") pod \"3b12d15f-129b-4121-85ac-c83c3581fcc2\" (UID: \"3b12d15f-129b-4121-85ac-c83c3581fcc2\") "
Nov 04 03:44:21.227157 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226894    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-model-cache\") pod \"f6339e78-6825-4246-a5d2-11831cd454af\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") "
Nov 04 03:44:21.227157 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226930    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-dshm\") pod \"f6339e78-6825-4246-a5d2-11831cd454af\" (UID: \"f6339e78-6825-4246-a5d2-11831cd454af\") "
Nov 04 03:44:21.227157 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.226970    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-model-cache\") pod \"1152bcd4-4f6e-4068-827d-02514564797d\" (UID: \"1152bcd4-4f6e-4068-827d-02514564797d\") "
Nov 04 03:44:21.227651 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.227619    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-model-cache" pod "1152bcd4-4f6e-4068-827d-02514564797d" (UID: "1152bcd4-4f6e-4068-827d-02514564797d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.229455 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.229087    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-model-cache" pod "5163cb68-1d76-43af-8654-990a89eb7405" (UID: "5163cb68-1d76-43af-8654-990a89eb7405"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.229455 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.229240    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-model-cache" pod "3b12d15f-129b-4121-85ac-c83c3581fcc2" (UID: "3b12d15f-129b-4121-85ac-c83c3581fcc2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.229653 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.229457    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-home" pod "3b12d15f-129b-4121-85ac-c83c3581fcc2" (UID: "3b12d15f-129b-4121-85ac-c83c3581fcc2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.229653 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.229462    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-tmp-dir" pod "f6339e78-6825-4246-a5d2-11831cd454af" (UID: "f6339e78-6825-4246-a5d2-11831cd454af"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.230069 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.229899    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-tmp-dir" pod "1152bcd4-4f6e-4068-827d-02514564797d" (UID: "1152bcd4-4f6e-4068-827d-02514564797d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.230069 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.229941    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-tmp-dir" pod "3b12d15f-129b-4121-85ac-c83c3581fcc2" (UID: "3b12d15f-129b-4121-85ac-c83c3581fcc2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.230069 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.229949    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-home" pod "5163cb68-1d76-43af-8654-990a89eb7405" (UID: "5163cb68-1d76-43af-8654-990a89eb7405"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.230425 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.230312    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-home" pod "1152bcd4-4f6e-4068-827d-02514564797d" (UID: "1152bcd4-4f6e-4068-827d-02514564797d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.230526 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.230423    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1152bcd4-4f6e-4068-827d-02514564797d-tls-certs" pod "1152bcd4-4f6e-4068-827d-02514564797d" (UID: "1152bcd4-4f6e-4068-827d-02514564797d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:44:21.231216 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.231053    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-model-cache" pod "f6339e78-6825-4246-a5d2-11831cd454af" (UID: "f6339e78-6825-4246-a5d2-11831cd454af"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.231216 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.231121    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-home" pod "f6339e78-6825-4246-a5d2-11831cd454af" (UID: "f6339e78-6825-4246-a5d2-11831cd454af"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.231216 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.231175    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-tmp-dir" pod "5163cb68-1d76-43af-8654-990a89eb7405" (UID: "5163cb68-1d76-43af-8654-990a89eb7405"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.231216 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.231197    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-dshm" pod "3b12d15f-129b-4121-85ac-c83c3581fcc2" (UID: "3b12d15f-129b-4121-85ac-c83c3581fcc2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.232258 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.232187    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b12d15f-129b-4121-85ac-c83c3581fcc2-tls-certs" pod "3b12d15f-129b-4121-85ac-c83c3581fcc2" (UID: "3b12d15f-129b-4121-85ac-c83c3581fcc2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:44:21.232258 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.232194    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1152bcd4-4f6e-4068-827d-02514564797d-kube-api-access-m276p" pod "1152bcd4-4f6e-4068-827d-02514564797d" (UID: "1152bcd4-4f6e-4068-827d-02514564797d"). InnerVolumeSpecName "kube-api-access-m276p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:21.232258 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.232210    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5163cb68-1d76-43af-8654-990a89eb7405-kube-api-access-6j5h5" pod "5163cb68-1d76-43af-8654-990a89eb7405" (UID: "5163cb68-1d76-43af-8654-990a89eb7405"). InnerVolumeSpecName "kube-api-access-6j5h5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:21.232258 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.232227    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b12d15f-129b-4121-85ac-c83c3581fcc2-kube-api-access-mqgwf" pod "3b12d15f-129b-4121-85ac-c83c3581fcc2" (UID: "3b12d15f-129b-4121-85ac-c83c3581fcc2"). InnerVolumeSpecName "kube-api-access-mqgwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:21.232510 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.232325    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-dshm" pod "1152bcd4-4f6e-4068-827d-02514564797d" (UID: "1152bcd4-4f6e-4068-827d-02514564797d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.232779 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.232644    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6339e78-6825-4246-a5d2-11831cd454af-kube-api-access-wbdq7" pod "f6339e78-6825-4246-a5d2-11831cd454af" (UID: "f6339e78-6825-4246-a5d2-11831cd454af"). InnerVolumeSpecName "kube-api-access-wbdq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:21.232779 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.232740    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5163cb68-1d76-43af-8654-990a89eb7405-tls-certs" pod "5163cb68-1d76-43af-8654-990a89eb7405" (UID: "5163cb68-1d76-43af-8654-990a89eb7405"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:44:21.232945 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.232824    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6339e78-6825-4246-a5d2-11831cd454af-tls-certs" pod "f6339e78-6825-4246-a5d2-11831cd454af" (UID: "f6339e78-6825-4246-a5d2-11831cd454af"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:44:21.233613 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.233590    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-dshm" pod "f6339e78-6825-4246-a5d2-11831cd454af" (UID: "f6339e78-6825-4246-a5d2-11831cd454af"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.234055 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.234040    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-dshm" pod "5163cb68-1d76-43af-8654-990a89eb7405" (UID: "5163cb68-1d76-43af-8654-990a89eb7405"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:21.328755 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328723    4420 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-home\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328755 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328753    4420 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-dshm\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328755 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328762    4420 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-home\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328770    4420 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-tmp-dir\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328778    4420 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-model-cache\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328788    4420 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-dshm\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328797    4420 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-model-cache\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328806    4420 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5163cb68-1d76-43af-8654-990a89eb7405-tls-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328814    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m276p\" (UniqueName: \"kubernetes.io/projected/1152bcd4-4f6e-4068-827d-02514564797d-kube-api-access-m276p\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328823    4420 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b12d15f-129b-4121-85ac-c83c3581fcc2-tls-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328831    4420 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6339e78-6825-4246-a5d2-11831cd454af-tls-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328860    4420 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-model-cache\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328871    4420 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-tmp-dir\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328880    4420 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-home\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328888    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6j5h5\" (UniqueName: \"kubernetes.io/projected/5163cb68-1d76-43af-8654-990a89eb7405-kube-api-access-6j5h5\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328896    4420 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6339e78-6825-4246-a5d2-11831cd454af-tmp-dir\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328904    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqgwf\" (UniqueName: \"kubernetes.io/projected/3b12d15f-129b-4121-85ac-c83c3581fcc2-kube-api-access-mqgwf\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328912    4420 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-model-cache\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328919    4420 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1152bcd4-4f6e-4068-827d-02514564797d-home\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328928    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbdq7\" (UniqueName: \"kubernetes.io/projected/f6339e78-6825-4246-a5d2-11831cd454af-kube-api-access-wbdq7\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328937    4420 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1152bcd4-4f6e-4068-827d-02514564797d-tls-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328945    4420 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b12d15f-129b-4121-85ac-c83c3581fcc2-dshm\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328953    4420 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-dshm\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.328996 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.328961    4420 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5163cb68-1d76-43af-8654-990a89eb7405-tmp-dir\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:21.370219 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.370131    4420 generic.go:356] "Generic (PLEG): container finished" podID="5163cb68-1d76-43af-8654-990a89eb7405" containerID="50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277" exitCode=0
Nov 04 03:44:21.370497 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.370379    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"
Nov 04 03:44:21.370716 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.370514    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg" event={"ID":"5163cb68-1d76-43af-8654-990a89eb7405","Type":"ContainerDied","Data":"50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277"}
Nov 04 03:44:21.370716 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.370559    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg" event={"ID":"5163cb68-1d76-43af-8654-990a89eb7405","Type":"ContainerDied","Data":"7ed8dc31444df0a5c84186868cbef7c7be0b97ac4ade26f3ec582b5d3dc52c16"}
Nov 04 03:44:21.370716 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.370581    4420 scope.go:122] "RemoveContainer" containerID="50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277"
Nov 04 03:44:21.375060 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.374838    4420 generic.go:356] "Generic (PLEG): container finished" podID="3b12d15f-129b-4121-85ac-c83c3581fcc2" containerID="fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3" exitCode=0
Nov 04 03:44:21.375060 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.374937    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"
Nov 04 03:44:21.375060 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.374984    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5" event={"ID":"3b12d15f-129b-4121-85ac-c83c3581fcc2","Type":"ContainerDied","Data":"fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3"}
Nov 04 03:44:21.375060 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.375018    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5" event={"ID":"3b12d15f-129b-4121-85ac-c83c3581fcc2","Type":"ContainerDied","Data":"387ee13811f28d1836dfd08af4be3151a49478f7738a6cc9db5af585bf9a50e3"}
Nov 04 03:44:21.378583 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.378557    4420 generic.go:356] "Generic (PLEG): container finished" podID="1152bcd4-4f6e-4068-827d-02514564797d" containerID="d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018" exitCode=0
Nov 04 03:44:21.378682 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.378640    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt" event={"ID":"1152bcd4-4f6e-4068-827d-02514564797d","Type":"ContainerDied","Data":"d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018"}
Nov 04 03:44:21.378682 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.378670    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt" event={"ID":"1152bcd4-4f6e-4068-827d-02514564797d","Type":"ContainerDied","Data":"83c30c27cb255deb90192f2c2b427258356ad873f5f24c4187f03ac42f22ee7a"}
Nov 04 03:44:21.378759 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.378735    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"
Nov 04 03:44:21.388269 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.388251    4420 scope.go:122] "RemoveContainer" containerID="50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277"
Nov 04 03:44:21.388883 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.388603    4420 generic.go:356] "Generic (PLEG): container finished" podID="f6339e78-6825-4246-a5d2-11831cd454af" containerID="46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e" exitCode=0
Nov 04 03:44:21.388883 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.388639    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"
Nov 04 03:44:21.388883 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.388688    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x" event={"ID":"f6339e78-6825-4246-a5d2-11831cd454af","Type":"ContainerDied","Data":"46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e"}
Nov 04 03:44:21.388883 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.388714    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x" event={"ID":"f6339e78-6825-4246-a5d2-11831cd454af","Type":"ContainerDied","Data":"8bf794205c1b7c0e2c540dcc71b9db8158c78c3a347e9c9a9a110a4a3887fb5f"}
Nov 04 03:44:21.393440 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:44:21.393408    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277\": container with ID starting with 50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277 not found: ID does not exist" containerID="50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277"
Nov 04 03:44:21.393539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.393447    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277"} err="failed to get container status \"50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277\": rpc error: code = NotFound desc = could not find container \"50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277\": container with ID starting with 50a7f51c54f6e023990d22a76362a952186a08a083d77d9bc2fb138c54c59277 not found: ID does not exist"
Nov 04 03:44:21.393539 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.393474    4420 scope.go:122] "RemoveContainer" containerID="fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3"
Nov 04 03:44:21.405632 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.405608    4420 scope.go:122] "RemoveContainer" containerID="fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3"
Nov 04 03:44:21.406133 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:44:21.406107    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3\": container with ID starting with fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3 not found: ID does not exist" containerID="fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3"
Nov 04 03:44:21.406251 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.406146    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3"} err="failed to get container status \"fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3\": rpc error: code = NotFound desc = could not find container \"fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3\": container with ID starting with fe8fcaad0b4c5400fc724d0aedfe5017678f7f359d5d329c43f38c0733c172f3 not found: ID does not exist"
Nov 04 03:44:21.406251 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.406165    4420 scope.go:122] "RemoveContainer" containerID="d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018"
Nov 04 03:44:21.417318 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.417293    4420 scope.go:122] "RemoveContainer" containerID="d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018"
Nov 04 03:44:21.420697 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:44:21.420662    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018\": container with ID starting with d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018 not found: ID does not exist" containerID="d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018"
Nov 04 03:44:21.420880 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.420704    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018"} err="failed to get container status \"d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018\": rpc error: code = NotFound desc = could not find container \"d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018\": container with ID starting with d493e4cc6ccce70af5714396a7a136d8b318e0414eebc31bf000a3d9c0a26018 not found: ID does not exist"
Nov 04 03:44:21.420880 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.420734    4420 scope.go:122] "RemoveContainer" containerID="46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e"
Nov 04 03:44:21.430083 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.430065    4420 scope.go:122] "RemoveContainer" containerID="46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e"
Nov 04 03:44:21.430439 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:44:21.430413    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e\": container with ID starting with 46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e not found: ID does not exist" containerID="46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e"
Nov 04 03:44:21.430616 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.430446    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e"} err="failed to get container status \"46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e\": rpc error: code = NotFound desc = could not find container \"46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e\": container with ID starting with 46ea7247a80e8f2edc96958188b679b213124ce17dc8a52069f337df0093fe2e not found: ID does not exist"
Nov 04 03:44:21.689236 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.689158    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"]
Nov 04 03:44:21.693604 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.693572    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/premium-w0-092f8c-kserve-5bc544cf6-kpqrg"]
Nov 04 03:44:21.796926 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.796893    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"]
Nov 04 03:44:21.801024 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.800999    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/distinct-2-w0-092f8c-kserve-85d6fd8676-gwdc5"]
Nov 04 03:44:21.902909 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.902878    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"]
Nov 04 03:44:21.907680 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:21.907652    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt"]
Nov 04 03:44:22.010515 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:22.010484    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"]
Nov 04 03:44:22.014407 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:22.014378    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-092f8c/distinct-w0-092f8c-kserve-7696458459-nmm5x"]
Nov 04 03:44:22.119783 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:22.119737    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-092f8c/simulator-w0-092f8c-kserve-5648878f9c-cb7dt" podUID="1152bcd4-4f6e-4068-827d-02514564797d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:44:22.309548 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:22.307395    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1152bcd4-4f6e-4068-827d-02514564797d" path="/var/lib/kubelet/pods/1152bcd4-4f6e-4068-827d-02514564797d/volumes"
Nov 04 03:44:22.309548 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:22.308037    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b12d15f-129b-4121-85ac-c83c3581fcc2" path="/var/lib/kubelet/pods/3b12d15f-129b-4121-85ac-c83c3581fcc2/volumes"
Nov 04 03:44:22.309548 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:22.308588    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5163cb68-1d76-43af-8654-990a89eb7405" path="/var/lib/kubelet/pods/5163cb68-1d76-43af-8654-990a89eb7405/volumes"
Nov 04 03:44:22.310871 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:22.310192    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6339e78-6825-4246-a5d2-11831cd454af" path="/var/lib/kubelet/pods/f6339e78-6825-4246-a5d2-11831cd454af/volumes"
Nov 04 03:44:24.661214 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:24.661173    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:25.239583 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:25.239550    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:29.063928 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:29.063894    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"]
Nov 04 03:44:29.110240 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:29.110209    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f3194190-whck8"]
Nov 04 03:44:30.231768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:30.231726    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:30.306973 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:30.306942    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40d929de-fa04-41b7-8514-893046288682" path="/var/lib/kubelet/pods/40d929de-fa04-41b7-8514-893046288682/volumes"
Nov 04 03:44:30.966076 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:30.966041    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:32.650746 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:44:32.650707    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:44:34.636990 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:34.636958    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:36.438050 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:36.438016    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:37.248252 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:37.248217    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:37.527341 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:37.527256    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:39.212871 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:39.212819    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:41.301474 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:41.301431    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:44:41.501678 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:41.501651    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:42.048587 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:42.048546    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:42.197861 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:42.197811    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:43.473541 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:43.473508    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:45.319173 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:45.319131    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:45.819301 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:45.819262    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"]
Nov 04 03:44:45.819517 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:45.819473    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47" podUID="fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" containerName="istio-proxy" containerID="cri-o://09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527" gracePeriod=30
Nov 04 03:44:45.833320 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:45.833293    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:46.047577 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:46.047544    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"]
Nov 04 03:44:46.050228 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:46.050202    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a343d6b-mfgqx"]
Nov 04 03:44:46.305766 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:46.305735    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7996fc27-5167-4b43-b003-b4686898bbc9" path="/var/lib/kubelet/pods/7996fc27-5167-4b43-b003-b4686898bbc9/volumes"
Nov 04 03:44:46.580411 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:46.580327    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:49.217546 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:49.217512    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:49.366665 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:49.366628    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:51.067703 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.067677    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:44:51.111963 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.111883    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-credential-socket\") pod \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") "
Nov 04 03:44:51.111963 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.111936    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-envoy\") pod \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") "
Nov 04 03:44:51.112153 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.111979    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-data\") pod \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") "
Nov 04 03:44:51.112153 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112014    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istiod-ca-cert\") pod \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") "
Nov 04 03:44:51.112153 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112051    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-socket\") pod \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") "
Nov 04 03:44:51.112153 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112072    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-certs\") pod \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") "
Nov 04 03:44:51.112153 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112104    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-token\" (UniqueName: \"kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-token\") pod \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") "
Nov 04 03:44:51.112153 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112132    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-podinfo\") pod \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") "
Nov 04 03:44:51.112458 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112156    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-kube-api-access-7t9q2\" (UniqueName: \"kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-kube-api-access-7t9q2\") pod \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\" (UID: \"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924\") "
Nov 04 03:44:51.112458 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112125    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-credential-socket" pod "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" (UID: "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.112458 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112267    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-data" pod "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" (UID: "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.112458 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112414    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-certs" pod "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" (UID: "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.112667 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112501    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istiod-ca-cert" pod "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" (UID: "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:44:51.112667 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112564    4420 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-credential-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.112667 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112588    4420 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-data\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.112667 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112604    4420 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.112819 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.112789    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-socket" pod "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" (UID: "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.114216 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.114188    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-podinfo" pod "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" (UID: "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:44:51.114343 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.114215    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-envoy" pod "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" (UID: "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:44:51.114581 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.114561    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-token" pod "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" (UID: "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:51.114662 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.114585    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-kube-api-access-7t9q2" pod "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" (UID: "fd1e4ba6-3c59-4786-ba2e-ffb5d0402924"). InnerVolumeSpecName "kube-api-access-7t9q2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:44:51.213584 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.213551    4420 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-envoy\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.213584 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.213581    4420 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istiod-ca-cert\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.213584 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.213592    4420 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-workload-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.213806 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.213608    4420 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-token\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.213806 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.213618    4420 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-istio-podinfo\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.213806 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.213627    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t9q2\" (UniqueName: \"kubernetes.io/projected/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924-kube-api-access-7t9q2\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:44:51.552219 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.552183    4420 generic.go:356] "Generic (PLEG): container finished" podID="fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" containerID="09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527" exitCode=0
Nov 04 03:44:51.552384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.552227    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"
Nov 04 03:44:51.552384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.552271    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47" event={"ID":"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924","Type":"ContainerDied","Data":"09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527"}
Nov 04 03:44:51.552384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.552329    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47" event={"ID":"fd1e4ba6-3c59-4786-ba2e-ffb5d0402924","Type":"ContainerDied","Data":"3a032f3c021331fd37f4229470a80f52cd07342dd733592a1529de136e460741"}
Nov 04 03:44:51.552384 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.552357    4420 scope.go:122] "RemoveContainer" containerID="09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527"
Nov 04 03:44:51.562147 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.562132    4420 scope.go:122] "RemoveContainer" containerID="09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527"
Nov 04 03:44:51.562413 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:44:51.562392    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527\": container with ID starting with 09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527 not found: ID does not exist" containerID="09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527"
Nov 04 03:44:51.562508 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.562420    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527"} err="failed to get container status \"09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527\": rpc error: code = NotFound desc = could not find container \"09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527\": container with ID starting with 09f747c88a09f40d38c817521c128a8c14b2d1d759e10bc57749ac26e8f30527 not found: ID does not exist"
Nov 04 03:44:51.707523 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.707496    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"]
Nov 04 03:44:51.711219 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.711199    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-092f8c-openshift-default-fffbc45cc-77c47"]
Nov 04 03:44:51.823552 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:51.823454    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:44:52.305689 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:52.305659    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" path="/var/lib/kubelet/pods/fd1e4ba6-3c59-4786-ba2e-ffb5d0402924/volumes"
Nov 04 03:44:56.849785 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:44:56.849748    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:00.173436 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.173397    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"]
Nov 04 03:45:00.174059 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174034    4420 cpu_manager.go:414] "containerMap: removing container" podUID="5163cb68-1d76-43af-8654-990a89eb7405" containerName="main"
Nov 04 03:45:00.174059 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174059    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5163cb68-1d76-43af-8654-990a89eb7405" containerName="main"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174072    4420 cpu_manager.go:414] "containerMap: removing container" podUID="301f8896-55ce-4194-8c92-89c352d19a4c" containerName="revoke-keys"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174080    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="301f8896-55ce-4194-8c92-89c352d19a4c" containerName="revoke-keys"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174096    4420 cpu_manager.go:414] "containerMap: removing container" podUID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" containerName="istio-proxy"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174103    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" containerName="istio-proxy"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174126    4420 cpu_manager.go:414] "containerMap: removing container" podUID="1152bcd4-4f6e-4068-827d-02514564797d" containerName="main"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174133    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1152bcd4-4f6e-4068-827d-02514564797d" containerName="main"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174143    4420 cpu_manager.go:414] "containerMap: removing container" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174150    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174181    4420 cpu_manager.go:414] "containerMap: removing container" podUID="85dd0445-88e0-4c88-8d9a-14fbf6bc2656" containerName="main"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174188    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85dd0445-88e0-4c88-8d9a-14fbf6bc2656" containerName="main"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174198    4420 cpu_manager.go:414] "containerMap: removing container" podUID="fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" containerName="istio-proxy"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174204    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" containerName="istio-proxy"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174214    4420 cpu_manager.go:414] "containerMap: removing container" podUID="3b12d15f-129b-4121-85ac-c83c3581fcc2" containerName="main"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174220    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b12d15f-129b-4121-85ac-c83c3581fcc2" containerName="main"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174242    4420 cpu_manager.go:414] "containerMap: removing container" podUID="8165ded7-cbdd-41e6-9790-06ad19ab9a2e" containerName="payload-processing"
Nov 04 03:45:00.174250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174249    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8165ded7-cbdd-41e6-9790-06ad19ab9a2e" containerName="payload-processing"
Nov 04 03:45:00.174715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174265    4420 cpu_manager.go:414] "containerMap: removing container" podUID="f6339e78-6825-4246-a5d2-11831cd454af" containerName="main"
Nov 04 03:45:00.174715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174272    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6339e78-6825-4246-a5d2-11831cd454af" containerName="main"
Nov 04 03:45:00.174715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174361    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="1152bcd4-4f6e-4068-827d-02514564797d" containerName="main"
Nov 04 03:45:00.174715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174374    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="156e2961-9ef5-43b5-8378-893caecef3d4" containerName="payload-processing"
Nov 04 03:45:00.174715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174385    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="85dd0445-88e0-4c88-8d9a-14fbf6bc2656" containerName="main"
Nov 04 03:45:00.174715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174394    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd1e4ba6-3c59-4786-ba2e-ffb5d0402924" containerName="istio-proxy"
Nov 04 03:45:00.174715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174401    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b12d15f-129b-4121-85ac-c83c3581fcc2" containerName="main"
Nov 04 03:45:00.174715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174411    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="8165ded7-cbdd-41e6-9790-06ad19ab9a2e" containerName="payload-processing"
Nov 04 03:45:00.174715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174421    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6339e78-6825-4246-a5d2-11831cd454af" containerName="main"
Nov 04 03:45:00.174715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174431    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="5163cb68-1d76-43af-8654-990a89eb7405" containerName="main"
Nov 04 03:45:00.174715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174439    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="301f8896-55ce-4194-8c92-89c352d19a4c" containerName="revoke-keys"
Nov 04 03:45:00.174715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.174446    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c75e12d-2787-43af-a6f2-1e678b9c33f3" containerName="istio-proxy"
Nov 04 03:45:00.179001 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.178984    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"]
Nov 04 03:45:00.179122 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.179108    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"
Nov 04 03:45:00.182269 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.182253    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"
Nov 04 03:45:00.228959 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.228929    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-x9cnx\""
Nov 04 03:45:00.228959 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.228928    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 03:45:00.229591 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.229571    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 03:45:00.279949 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.279919    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"]
Nov 04 03:45:00.282226 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.282202    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"]
Nov 04 03:45:00.299565 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.299529    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856ee09f-b9e3-46d2-9d2c-c49ea8000274-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb\" (UID: \"856ee09f-b9e3-46d2-9d2c-c49ea8000274\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"
Nov 04 03:45:00.299739 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.299612    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j\" (UID: \"c0098b4f-d0d9-4de0-b10c-871e60a48ef9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"
Nov 04 03:45:00.299739 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.299649    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqmtr\" (UniqueName: \"kubernetes.io/projected/856ee09f-b9e3-46d2-9d2c-c49ea8000274-kube-api-access-hqmtr\") pod \"maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb\" (UID: \"856ee09f-b9e3-46d2-9d2c-c49ea8000274\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"
Nov 04 03:45:00.299891 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.299824    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glstr\" (UniqueName: \"kubernetes.io/projected/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-kube-api-access-glstr\") pod \"maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j\" (UID: \"c0098b4f-d0d9-4de0-b10c-871e60a48ef9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"
Nov 04 03:45:00.400707 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.400674    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856ee09f-b9e3-46d2-9d2c-c49ea8000274-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb\" (UID: \"856ee09f-b9e3-46d2-9d2c-c49ea8000274\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"
Nov 04 03:45:00.400707 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.400708    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j\" (UID: \"c0098b4f-d0d9-4de0-b10c-871e60a48ef9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"
Nov 04 03:45:00.400947 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.400728    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqmtr\" (UniqueName: \"kubernetes.io/projected/856ee09f-b9e3-46d2-9d2c-c49ea8000274-kube-api-access-hqmtr\") pod \"maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb\" (UID: \"856ee09f-b9e3-46d2-9d2c-c49ea8000274\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"
Nov 04 03:45:00.400947 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.400824    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glstr\" (UniqueName: \"kubernetes.io/projected/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-kube-api-access-glstr\") pod \"maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j\" (UID: \"c0098b4f-d0d9-4de0-b10c-871e60a48ef9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"
Nov 04 03:45:00.401363 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.401343    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j\" (UID: \"c0098b4f-d0d9-4de0-b10c-871e60a48ef9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"
Nov 04 03:45:00.401404 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.401382    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856ee09f-b9e3-46d2-9d2c-c49ea8000274-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb\" (UID: \"856ee09f-b9e3-46d2-9d2c-c49ea8000274\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"
Nov 04 03:45:00.487288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.487212    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqmtr\" (UniqueName: \"kubernetes.io/projected/856ee09f-b9e3-46d2-9d2c-c49ea8000274-kube-api-access-hqmtr\") pod \"maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb\" (UID: \"856ee09f-b9e3-46d2-9d2c-c49ea8000274\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"
Nov 04 03:45:00.492635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.492608    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"
Nov 04 03:45:00.502418 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.502394    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glstr\" (UniqueName: \"kubernetes.io/projected/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-kube-api-access-glstr\") pod \"maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j\" (UID: \"c0098b4f-d0d9-4de0-b10c-871e60a48ef9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"
Nov 04 03:45:00.799317 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.799242    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"
Nov 04 03:45:00.862516 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:00.862484    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"]
Nov 04 03:45:00.867201 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:45:00.867169    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod856ee09f_b9e3_46d2_9d2c_c49ea8000274.slice/crio-cb7cce69efb93d7c803ace6bf4807222eb57b111dcf2be3578ee7a9f55af9c04 WatchSource:0}: Error finding container cb7cce69efb93d7c803ace6bf4807222eb57b111dcf2be3578ee7a9f55af9c04: Status 404 returned error can't find the container with id cb7cce69efb93d7c803ace6bf4807222eb57b111dcf2be3578ee7a9f55af9c04
Nov 04 03:45:01.168133 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:01.168101    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"]
Nov 04 03:45:01.174733 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:45:01.174707    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0098b4f_d0d9_4de0_b10c_871e60a48ef9.slice/crio-66b6dbc773e5065297669e4826ccc583da39b63cbe2ff939a6f9c3ea8eee4e20 WatchSource:0}: Error finding container 66b6dbc773e5065297669e4826ccc583da39b63cbe2ff939a6f9c3ea8eee4e20: Status 404 returned error can't find the container with id 66b6dbc773e5065297669e4826ccc583da39b63cbe2ff939a6f9c3ea8eee4e20
Nov 04 03:45:01.602421 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:01.602386    4420 generic.go:356] "Generic (PLEG): container finished" podID="856ee09f-b9e3-46d2-9d2c-c49ea8000274" containerID="78cafa91da712df0c3320f56122f435f4085bdbbf10af82634d692607c826ed9" exitCode=60
Nov 04 03:45:01.602577 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:01.602469    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb" event={"ID":"856ee09f-b9e3-46d2-9d2c-c49ea8000274","Type":"ContainerDied","Data":"78cafa91da712df0c3320f56122f435f4085bdbbf10af82634d692607c826ed9"}
Nov 04 03:45:01.602577 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:01.602523    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb" event={"ID":"856ee09f-b9e3-46d2-9d2c-c49ea8000274","Type":"ContainerStarted","Data":"cb7cce69efb93d7c803ace6bf4807222eb57b111dcf2be3578ee7a9f55af9c04"}
Nov 04 03:45:01.602710 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:01.602696    4420 scope.go:122] "RemoveContainer" containerID="78cafa91da712df0c3320f56122f435f4085bdbbf10af82634d692607c826ed9"
Nov 04 03:45:01.604270 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:01.604242    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j" event={"ID":"c0098b4f-d0d9-4de0-b10c-871e60a48ef9","Type":"ContainerStarted","Data":"e5662e9a0529393e198a0318c2d9beccc6d40e0020f4b66e63c2c88768c4d341"}
Nov 04 03:45:01.604372 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:01.604292    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j" event={"ID":"c0098b4f-d0d9-4de0-b10c-871e60a48ef9","Type":"ContainerStarted","Data":"66b6dbc773e5065297669e4826ccc583da39b63cbe2ff939a6f9c3ea8eee4e20"}
Nov 04 03:45:01.730021 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:01.729979    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:01.828271 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:01.828218    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j" podStartSLOduration=1.828203297 podStartE2EDuration="1.828203297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:45:01.826454152 +0000 UTC m=+1671.641245431" watchObservedRunningTime="2026-11-04 03:45:01.828203297 +0000 UTC m=+1671.642994574"
Nov 04 03:45:02.611003 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:02.610911    4420 generic.go:356] "Generic (PLEG): container finished" podID="856ee09f-b9e3-46d2-9d2c-c49ea8000274" containerID="6ff6f82e394eb08e21697225e10abb8edc35a90fadaeebd59ca0f47a5c311d59" exitCode=60
Nov 04 03:45:02.611003 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:02.610991    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb" event={"ID":"856ee09f-b9e3-46d2-9d2c-c49ea8000274","Type":"ContainerDied","Data":"6ff6f82e394eb08e21697225e10abb8edc35a90fadaeebd59ca0f47a5c311d59"}
Nov 04 03:45:02.611527 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:02.611052    4420 scope.go:122] "RemoveContainer" containerID="78cafa91da712df0c3320f56122f435f4085bdbbf10af82634d692607c826ed9"
Nov 04 03:45:02.611527 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:02.611233    4420 scope.go:122] "RemoveContainer" containerID="6ff6f82e394eb08e21697225e10abb8edc35a90fadaeebd59ca0f47a5c311d59"
Nov 04 03:45:02.611527 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:45:02.611492    4420 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb_odh-ai-gateway-infra(856ee09f-b9e3-46d2-9d2c-c49ea8000274)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb" podUID="856ee09f-b9e3-46d2-9d2c-c49ea8000274"
Nov 04 03:45:02.612655 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:02.612635    4420 generic.go:356] "Generic (PLEG): container finished" podID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9" containerID="e5662e9a0529393e198a0318c2d9beccc6d40e0020f4b66e63c2c88768c4d341" exitCode=60
Nov 04 03:45:02.612759 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:02.612702    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j" event={"ID":"c0098b4f-d0d9-4de0-b10c-871e60a48ef9","Type":"ContainerDied","Data":"e5662e9a0529393e198a0318c2d9beccc6d40e0020f4b66e63c2c88768c4d341"}
Nov 04 03:45:02.612924 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:02.612910    4420 scope.go:122] "RemoveContainer" containerID="e5662e9a0529393e198a0318c2d9beccc6d40e0020f4b66e63c2c88768c4d341"
Nov 04 03:45:02.667374 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:45:02.667344    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:45:03.619760 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:03.619674    4420 scope.go:122] "RemoveContainer" containerID="6ff6f82e394eb08e21697225e10abb8edc35a90fadaeebd59ca0f47a5c311d59"
Nov 04 03:45:03.620184 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:45:03.619943    4420 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb_odh-ai-gateway-infra(856ee09f-b9e3-46d2-9d2c-c49ea8000274)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb" podUID="856ee09f-b9e3-46d2-9d2c-c49ea8000274"
Nov 04 03:45:03.621228 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:03.621207    4420 generic.go:356] "Generic (PLEG): container finished" podID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9" containerID="67c0343bccf66e83ef9979ba45c2cbac3d1545714636ee56522eb007ce1ecd2c" exitCode=60
Nov 04 03:45:03.621328 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:03.621261    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j" event={"ID":"c0098b4f-d0d9-4de0-b10c-871e60a48ef9","Type":"ContainerDied","Data":"67c0343bccf66e83ef9979ba45c2cbac3d1545714636ee56522eb007ce1ecd2c"}
Nov 04 03:45:03.621328 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:03.621302    4420 scope.go:122] "RemoveContainer" containerID="e5662e9a0529393e198a0318c2d9beccc6d40e0020f4b66e63c2c88768c4d341"
Nov 04 03:45:03.621529 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:03.621503    4420 scope.go:122] "RemoveContainer" containerID="67c0343bccf66e83ef9979ba45c2cbac3d1545714636ee56522eb007ce1ecd2c"
Nov 04 03:45:03.621721 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:45:03.621703    4420 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j_odh-ai-gateway-infra(c0098b4f-d0d9-4de0-b10c-871e60a48ef9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j" podUID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9"
Nov 04 03:45:04.627557 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:04.627527    4420 scope.go:122] "RemoveContainer" containerID="67c0343bccf66e83ef9979ba45c2cbac3d1545714636ee56522eb007ce1ecd2c"
Nov 04 03:45:04.627982 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:45:04.627755    4420 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j_odh-ai-gateway-infra(c0098b4f-d0d9-4de0-b10c-871e60a48ef9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j" podUID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9"
Nov 04 03:45:06.470659 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:06.470619    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:06.616271 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:06.616231    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:07.355157 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:07.355120    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:09.357958 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:09.357923    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:11.524250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:11.524216    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:14.300416 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:14.300379    4420 scope.go:122] "RemoveContainer" containerID="67c0343bccf66e83ef9979ba45c2cbac3d1545714636ee56522eb007ce1ecd2c"
Nov 04 03:45:14.678826 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:14.678796    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j" event={"ID":"c0098b4f-d0d9-4de0-b10c-871e60a48ef9","Type":"ContainerStarted","Data":"31bed6a2adcd1d9dbbdf12cf06bc139891517b6500871b75ddbd6a58766c57fc"}
Nov 04 03:45:15.300185 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:15.300155    4420 scope.go:122] "RemoveContainer" containerID="6ff6f82e394eb08e21697225e10abb8edc35a90fadaeebd59ca0f47a5c311d59"
Nov 04 03:45:15.685941 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:15.685854    4420 generic.go:356] "Generic (PLEG): container finished" podID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9" containerID="31bed6a2adcd1d9dbbdf12cf06bc139891517b6500871b75ddbd6a58766c57fc" exitCode=60
Nov 04 03:45:15.685941 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:15.685906    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j" event={"ID":"c0098b4f-d0d9-4de0-b10c-871e60a48ef9","Type":"ContainerDied","Data":"31bed6a2adcd1d9dbbdf12cf06bc139891517b6500871b75ddbd6a58766c57fc"}
Nov 04 03:45:15.686463 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:15.685967    4420 scope.go:122] "RemoveContainer" containerID="67c0343bccf66e83ef9979ba45c2cbac3d1545714636ee56522eb007ce1ecd2c"
Nov 04 03:45:15.686463 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:15.686306    4420 scope.go:122] "RemoveContainer" containerID="31bed6a2adcd1d9dbbdf12cf06bc139891517b6500871b75ddbd6a58766c57fc"
Nov 04 03:45:15.686612 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:45:15.686589    4420 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j_odh-ai-gateway-infra(c0098b4f-d0d9-4de0-b10c-871e60a48ef9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j" podUID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9"
Nov 04 03:45:15.687939 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:15.687919    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb" event={"ID":"856ee09f-b9e3-46d2-9d2c-c49ea8000274","Type":"ContainerStarted","Data":"881a4e33836de7194f6d742b1935fcfac70f3e98778c743f8ab0019bdf368300"}
Nov 04 03:45:15.801682 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:15.801642    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"]
Nov 04 03:45:15.904418 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:15.904366    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb" podStartSLOduration=15.90435045 podStartE2EDuration="15.90435045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:45:15.903459315 +0000 UTC m=+1685.718250596" watchObservedRunningTime="2026-11-04 03:45:15.90435045 +0000 UTC m=+1685.719141727"
Nov 04 03:45:16.432835 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:16.432692    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:16.695890 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:16.695828    4420 generic.go:356] "Generic (PLEG): container finished" podID="856ee09f-b9e3-46d2-9d2c-c49ea8000274" containerID="881a4e33836de7194f6d742b1935fcfac70f3e98778c743f8ab0019bdf368300" exitCode=60
Nov 04 03:45:16.695890 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:16.695871    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb" event={"ID":"856ee09f-b9e3-46d2-9d2c-c49ea8000274","Type":"ContainerDied","Data":"881a4e33836de7194f6d742b1935fcfac70f3e98778c743f8ab0019bdf368300"}
Nov 04 03:45:16.696444 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:16.695924    4420 scope.go:122] "RemoveContainer" containerID="6ff6f82e394eb08e21697225e10abb8edc35a90fadaeebd59ca0f47a5c311d59"
Nov 04 03:45:16.696444 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:16.696244    4420 scope.go:122] "RemoveContainer" containerID="881a4e33836de7194f6d742b1935fcfac70f3e98778c743f8ab0019bdf368300"
Nov 04 03:45:16.696549 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:45:16.696481    4420 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb_odh-ai-gateway-infra(856ee09f-b9e3-46d2-9d2c-c49ea8000274)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb" podUID="856ee09f-b9e3-46d2-9d2c-c49ea8000274"
Nov 04 03:45:16.834860 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:16.834818    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"
Nov 04 03:45:16.911823 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:16.911789    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"]
Nov 04 03:45:16.955325 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:16.955227    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-maas-api-service-ca\") pod \"c0098b4f-d0d9-4de0-b10c-871e60a48ef9\" (UID: \"c0098b4f-d0d9-4de0-b10c-871e60a48ef9\") "
Nov 04 03:45:16.955325 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:16.955313    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-kube-api-access-glstr\" (UniqueName: \"kubernetes.io/projected/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-kube-api-access-glstr\") pod \"c0098b4f-d0d9-4de0-b10c-871e60a48ef9\" (UID: \"c0098b4f-d0d9-4de0-b10c-871e60a48ef9\") "
Nov 04 03:45:16.955626 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:16.955601    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-maas-api-service-ca" pod "c0098b4f-d0d9-4de0-b10c-871e60a48ef9" (UID: "c0098b4f-d0d9-4de0-b10c-871e60a48ef9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:45:16.957275 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:16.957252    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-kube-api-access-glstr" pod "c0098b4f-d0d9-4de0-b10c-871e60a48ef9" (UID: "c0098b4f-d0d9-4de0-b10c-871e60a48ef9"). InnerVolumeSpecName "kube-api-access-glstr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:45:17.057134 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:17.057094    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glstr\" (UniqueName: \"kubernetes.io/projected/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-kube-api-access-glstr\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:45:17.057134 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:17.057133    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0098b4f-d0d9-4de0-b10c-871e60a48ef9-maas-api-service-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:45:17.707208 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:17.707176    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j" event={"ID":"c0098b4f-d0d9-4de0-b10c-871e60a48ef9","Type":"ContainerDied","Data":"66b6dbc773e5065297669e4826ccc583da39b63cbe2ff939a6f9c3ea8eee4e20"}
Nov 04 03:45:17.707607 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:17.707242    4420 scope.go:122] "RemoveContainer" containerID="31bed6a2adcd1d9dbbdf12cf06bc139891517b6500871b75ddbd6a58766c57fc"
Nov 04 03:45:17.707607 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:17.707337    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"
Nov 04 03:45:17.848367 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:17.848346    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"
Nov 04 03:45:17.920206 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:17.920185    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"]
Nov 04 03:45:17.924658 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:17.924636    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-fba151-29896065-ng69j"]
Nov 04 03:45:17.965648 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:17.965580    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/856ee09f-b9e3-46d2-9d2c-c49ea8000274-kube-api-access-hqmtr\" (UniqueName: \"kubernetes.io/projected/856ee09f-b9e3-46d2-9d2c-c49ea8000274-kube-api-access-hqmtr\") pod \"856ee09f-b9e3-46d2-9d2c-c49ea8000274\" (UID: \"856ee09f-b9e3-46d2-9d2c-c49ea8000274\") "
Nov 04 03:45:17.965731 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:17.965705    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/856ee09f-b9e3-46d2-9d2c-c49ea8000274-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856ee09f-b9e3-46d2-9d2c-c49ea8000274-maas-api-service-ca\") pod \"856ee09f-b9e3-46d2-9d2c-c49ea8000274\" (UID: \"856ee09f-b9e3-46d2-9d2c-c49ea8000274\") "
Nov 04 03:45:17.966069 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:17.966049    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/856ee09f-b9e3-46d2-9d2c-c49ea8000274-maas-api-service-ca" pod "856ee09f-b9e3-46d2-9d2c-c49ea8000274" (UID: "856ee09f-b9e3-46d2-9d2c-c49ea8000274"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:45:17.967586 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:17.967567    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/856ee09f-b9e3-46d2-9d2c-c49ea8000274-kube-api-access-hqmtr" pod "856ee09f-b9e3-46d2-9d2c-c49ea8000274" (UID: "856ee09f-b9e3-46d2-9d2c-c49ea8000274"). InnerVolumeSpecName "kube-api-access-hqmtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:45:18.067255 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:18.067219    4420 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856ee09f-b9e3-46d2-9d2c-c49ea8000274-maas-api-service-ca\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:45:18.067408 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:18.067258    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqmtr\" (UniqueName: \"kubernetes.io/projected/856ee09f-b9e3-46d2-9d2c-c49ea8000274-kube-api-access-hqmtr\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:45:18.306950 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:18.306907    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9" path="/var/lib/kubelet/pods/c0098b4f-d0d9-4de0-b10c-871e60a48ef9/volumes"
Nov 04 03:45:18.713732 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:18.713648    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"
Nov 04 03:45:18.713732 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:18.713676    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb" event={"ID":"856ee09f-b9e3-46d2-9d2c-c49ea8000274","Type":"ContainerDied","Data":"cb7cce69efb93d7c803ace6bf4807222eb57b111dcf2be3578ee7a9f55af9c04"}
Nov 04 03:45:18.714273 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:18.713744    4420 scope.go:122] "RemoveContainer" containerID="881a4e33836de7194f6d742b1935fcfac70f3e98778c743f8ab0019bdf368300"
Nov 04 03:45:18.869636 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:18.869592    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"]
Nov 04 03:45:18.916543 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:18.916509    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-951893-29896065-rvjtb"]
Nov 04 03:45:20.306383 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:20.306350    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="856ee09f-b9e3-46d2-9d2c-c49ea8000274" path="/var/lib/kubelet/pods/856ee09f-b9e3-46d2-9d2c-c49ea8000274/volumes"
Nov 04 03:45:21.325554 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:21.325521    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:26.127752 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:26.127714    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:31.010204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:31.010165    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:32.650067 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:45:32.650018    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:45:35.948473 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:35.948436    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:40.813056 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:40.813016    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:45.718293 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:45.718250    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:50.621387 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:50.621332    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:45:54.047551 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:54.047518    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"]
Nov 04 03:45:54.094208 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:54.094169    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec3256c7-pd5p2"]
Nov 04 03:45:54.306095 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:54.306017    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ad0f554-b218-4d84-9c72-6ca76a6361a7" path="/var/lib/kubelet/pods/7ad0f554-b218-4d84-9c72-6ca76a6361a7/volumes"
Nov 04 03:45:55.519962 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:45:55.519925    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:00.416005 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:00.415961    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:02.649426 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:46:02.649387    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:46:05.244620 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:05.244580    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:09.715989 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:09.715951    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:10.099086 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:10.099055    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:10.657366 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:10.657335    4420 scope.go:122] "RemoveContainer" containerID="e905d9d844b406853fcb01ff11a6d2d737ce77c764fccde65ceb461107623b28"
Nov 04 03:46:10.665419 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:10.665400    4420 scope.go:122] "RemoveContainer" containerID="cc30605ba16218069f0838db5a407b55838d1a85ddaa65e6e5b38fef2b9c6d54"
Nov 04 03:46:10.672819 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:10.672803    4420 scope.go:122] "RemoveContainer" containerID="ba2d8384dfa0b16b6a4d98c02bb13e46e302b7d82b30093b1380c64277ba77a2"
Nov 04 03:46:15.007424 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:15.007386    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:19.049396 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:19.049363    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"]
Nov 04 03:46:19.095991 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:19.095965    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20735c2f-rf2q9"]
Nov 04 03:46:19.866378 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:19.866333    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:20.306834 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:20.306801    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d489cdc1-ec1b-47f8-a09b-b25c09ee7dff" path="/var/lib/kubelet/pods/d489cdc1-ec1b-47f8-a09b-b25c09ee7dff/volumes"
Nov 04 03:46:24.708496 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:24.708401    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:29.515893 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:29.515832    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:32.649433 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:46:32.649391    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:46:34.428799 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:34.428755    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:39.231400 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:39.231361    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:44.204439 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:44.204396    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:49.026865 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:49.026800    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:53.913804 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:53.913770    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:46:58.818473 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:46:58.818433    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:02.648690 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:47:02.648653    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:47:03.712005 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:03.711976    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:08.531047 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:08.531007    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:08.676061 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:08.676005    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:10.058060 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:10.058015    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:10.488649 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:10.488622    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:47:10.488998 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:10.488980    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:47:10.707245 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:10.707219    4420 scope.go:122] "RemoveContainer" containerID="8b3b65ce2cdf628deeaed36293d2611e1ac103788996b7f39026aa742a9eb660"
Nov 04 03:47:13.395769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:13.395729    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:18.309012 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:18.308973    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:23.227704 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:23.227639    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:28.175188 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:28.175154    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:32.648729 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:47:32.648677    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:47:33.014014 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:33.013981    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:37.044826 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:37.044790    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"]
Nov 04 03:47:37.091340 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:37.091311    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-53dc6c4c-zf96d"]
Nov 04 03:47:37.921217 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:37.921177    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:38.305582 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:38.305553    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="301f8896-55ce-4194-8c92-89c352d19a4c" path="/var/lib/kubelet/pods/301f8896-55ce-4194-8c92-89c352d19a4c/volumes"
Nov 04 03:47:42.731131 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:42.731100    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:47.610679 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:47.610641    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:52.521417 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:52.521330    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:47:57.331701 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:47:57.331658    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:02.259818 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:02.259765    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:02.650186 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:48:02.650091    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:48:07.012010 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:07.011975    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:10.427143 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:10.427106    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:10.727476 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:10.727400    4420 scope.go:122] "RemoveContainer" containerID="990c8deefc620c1f17d1b7ac4a6b471226070b46464c0b1e3e3bef48f59fda74"
Nov 04 03:48:11.907610 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:11.907570    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:16.736758 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:16.736725    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:21.511522 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:21.511488    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:26.399781 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:26.399746    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:31.311979 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:31.311949    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:32.648116 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:48:32.648079    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:48:36.215107 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:36.215070    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:37.773867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:37.773810    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:41.111549 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:41.111498    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:45.927199 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:45.927164    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:50.799235 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:50.799205    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:53.515641 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:53.515610    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:48:55.627183 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:48:55.627152    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:00.540919 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:00.540885    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:02.648701 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:49:02.648668    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:49:04.528592 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:04.528557    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:04.683419 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:04.683375    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:05.033385 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:05.033349    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:05.427432 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:05.427343    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:09.382269 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:09.382233    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:49:09.871328 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:09.871289    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:10.324501 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:10.324465    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:10.745719 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:10.745683    4420 scope.go:122] "RemoveContainer" containerID="97691b32ac623d9f76091797aaf27de46d25962c0774ec31511930d82738cea6"
Nov 04 03:49:10.784390 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:10.784335    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:10.946715 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:10.946682    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:15.307771 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:15.307733    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:16.926103 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:16.926065    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:20.431108 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:20.431073    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:23.043993 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:23.043903    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:23.658492 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:23.658462    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"]
Nov 04 03:49:24.028171 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:24.028140    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:24.180082 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:24.180052    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:25.437777 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:25.437730    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:28.659393 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:28.659327    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t" podUID="4dd3b4c8-659f-49b2-99f8-13355a4ede22" containerName="payload-processing" containerID="cri-o://707f5b5ad54487e09c22973d88cea216bd2ed3442b5f9efe13c5ff25d0cbc2fb" gracePeriod=25
Nov 04 03:49:28.894393 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:28.894366    4420 generic.go:356] "Generic (PLEG): container finished" podID="4dd3b4c8-659f-49b2-99f8-13355a4ede22" containerID="707f5b5ad54487e09c22973d88cea216bd2ed3442b5f9efe13c5ff25d0cbc2fb" exitCode=0
Nov 04 03:49:28.894535 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:28.894440    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t" event={"ID":"4dd3b4c8-659f-49b2-99f8-13355a4ede22","Type":"ContainerDied","Data":"707f5b5ad54487e09c22973d88cea216bd2ed3442b5f9efe13c5ff25d0cbc2fb"}
Nov 04 03:49:28.894535 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:28.894500    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t" event={"ID":"4dd3b4c8-659f-49b2-99f8-13355a4ede22","Type":"ContainerDied","Data":"9b45d7213c1f7d1f3cc4d80b7b8929f77d9c5cbde3262612e56c72d08cb085e8"}
Nov 04 03:49:28.894535 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:28.894524    4420 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b45d7213c1f7d1f3cc4d80b7b8929f77d9c5cbde3262612e56c72d08cb085e8"
Nov 04 03:49:28.901684 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:28.901664    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"
Nov 04 03:49:28.976029 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:28.975971    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4dd3b4c8-659f-49b2-99f8-13355a4ede22-kube-api-access-kjlvn\" (UniqueName: \"kubernetes.io/projected/4dd3b4c8-659f-49b2-99f8-13355a4ede22-kube-api-access-kjlvn\") pod \"4dd3b4c8-659f-49b2-99f8-13355a4ede22\" (UID: \"4dd3b4c8-659f-49b2-99f8-13355a4ede22\") "
Nov 04 03:49:28.976137 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:28.976037    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4dd3b4c8-659f-49b2-99f8-13355a4ede22-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd3b4c8-659f-49b2-99f8-13355a4ede22-plugins-config-volume\") pod \"4dd3b4c8-659f-49b2-99f8-13355a4ede22\" (UID: \"4dd3b4c8-659f-49b2-99f8-13355a4ede22\") "
Nov 04 03:49:28.976380 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:28.976358    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dd3b4c8-659f-49b2-99f8-13355a4ede22-plugins-config-volume" pod "4dd3b4c8-659f-49b2-99f8-13355a4ede22" (UID: "4dd3b4c8-659f-49b2-99f8-13355a4ede22"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:49:28.976511 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:28.976496    4420 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd3b4c8-659f-49b2-99f8-13355a4ede22-plugins-config-volume\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:28.978103 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:28.978081    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dd3b4c8-659f-49b2-99f8-13355a4ede22-kube-api-access-kjlvn" pod "4dd3b4c8-659f-49b2-99f8-13355a4ede22" (UID: "4dd3b4c8-659f-49b2-99f8-13355a4ede22"). InnerVolumeSpecName "kube-api-access-kjlvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:29.077527 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:29.077496    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjlvn\" (UniqueName: \"kubernetes.io/projected/4dd3b4c8-659f-49b2-99f8-13355a4ede22-kube-api-access-kjlvn\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:29.260663 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:29.260631    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:29.778102 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:29.778071    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:29.900223 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:29.900193    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"
Nov 04 03:49:30.056882 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:30.056783    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"]
Nov 04 03:49:30.103561 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:30.103530    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6d170d-7cb75b8cb8-dkz4t"]
Nov 04 03:49:30.305186 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:30.305148    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4dd3b4c8-659f-49b2-99f8-13355a4ede22" path="/var/lib/kubelet/pods/4dd3b4c8-659f-49b2-99f8-13355a4ede22/volumes"
Nov 04 03:49:31.635218 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:31.635179    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:32.651013 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:49:32.650978    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:49:35.499516 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:35.499486    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:35.654971 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:35.654938    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:36.725130 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:36.725095    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:37.225461 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:37.225421    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:37.725266 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:37.725233    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:49:37.920250 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:37.920210    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:38.713573 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:38.713536    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:42.112321 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:42.112289    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:44.969106 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:44.969073    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:45.530488 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:45.530455    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"]
Nov 04 03:49:45.536717 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:45.536688    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:45.904510 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:45.904426    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:46.108103 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:46.108065    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:46.552920 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:46.552886    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:48.514582 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:48.514549    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:50.116860 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.116812    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:49:50.531172 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.531118    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv" podUID="9aa6de23-5c26-4d8d-acc8-03f6963b1dcd" containerName="payload-processing" containerID="cri-o://6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98" gracePeriod=25
Nov 04 03:49:50.773436 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.773411    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"
Nov 04 03:49:50.813777 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.813715    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:50.877213 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.877185    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-kube-api-access-ddbtl\" (UniqueName: \"kubernetes.io/projected/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-kube-api-access-ddbtl\") pod \"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd\" (UID: \"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd\") "
Nov 04 03:49:50.877401 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.877230    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-plugins-config-volume\") pod \"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd\" (UID: \"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd\") "
Nov 04 03:49:50.877659 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.877637    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-plugins-config-volume" pod "9aa6de23-5c26-4d8d-acc8-03f6963b1dcd" (UID: "9aa6de23-5c26-4d8d-acc8-03f6963b1dcd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:49:50.879313 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.879291    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-kube-api-access-ddbtl" pod "9aa6de23-5c26-4d8d-acc8-03f6963b1dcd" (UID: "9aa6de23-5c26-4d8d-acc8-03f6963b1dcd"). InnerVolumeSpecName "kube-api-access-ddbtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:50.978985 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.978950    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddbtl\" (UniqueName: \"kubernetes.io/projected/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-kube-api-access-ddbtl\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:50.978985 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.978978    4420 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd-plugins-config-volume\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:50.993813 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.993780    4420 generic.go:356] "Generic (PLEG): container finished" podID="9aa6de23-5c26-4d8d-acc8-03f6963b1dcd" containerID="6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98" exitCode=0
Nov 04 03:49:50.993961 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.993828    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"
Nov 04 03:49:50.993961 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.993869    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv" event={"ID":"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd","Type":"ContainerDied","Data":"6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98"}
Nov 04 03:49:50.993961 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.993920    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv" event={"ID":"9aa6de23-5c26-4d8d-acc8-03f6963b1dcd","Type":"ContainerDied","Data":"719accd26a892947d41c2bd1740fb5219e94412916bbaa6c8fe0739905dff85a"}
Nov 04 03:49:50.993961 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:50.993945    4420 scope.go:122] "RemoveContainer" containerID="6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98"
Nov 04 03:49:51.004598 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:51.004578    4420 scope.go:122] "RemoveContainer" containerID="6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98"
Nov 04 03:49:51.004854 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:49:51.004825    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98\": container with ID starting with 6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98 not found: ID does not exist" containerID="6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98"
Nov 04 03:49:51.004928 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:51.004883    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98"} err="failed to get container status \"6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98\": rpc error: code = NotFound desc = could not find container \"6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98\": container with ID starting with 6358d75c816f5470ed791f4bfe2e9a09fcd823d662e77d2be085a59f5ec8eb98 not found: ID does not exist"
Nov 04 03:49:51.015706 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:51.015679    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:51.150635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:51.150568    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"]
Nov 04 03:49:51.153137 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:51.153114    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-951893-5d6556fd68-p6pfv"]
Nov 04 03:49:52.305692 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:52.305663    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9aa6de23-5c26-4d8d-acc8-03f6963b1dcd" path="/var/lib/kubelet/pods/9aa6de23-5c26-4d8d-acc8-03f6963b1dcd/volumes"
Nov 04 03:49:52.399802 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:52.399774    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:52.905665 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:52.905637    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"]
Nov 04 03:49:52.905881 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:52.905836    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" podUID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerName="istio-proxy" containerID="cri-o://feaf5ad24eb85657a65733f4ebe275bc6ecf93113201c2585896bbdf3520f6a9" gracePeriod=30
Nov 04 03:49:52.926029 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:52.926000    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:53.529373 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:53.529335    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:53.730757 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:53.730710    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:56.517466 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:56.517423    4420 patch_prober.go:29] interesting pod/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:49:56.517964 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:56.517475    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" podUID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:49:58.040782 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.040750    4420 generic.go:356] "Generic (PLEG): container finished" podID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerID="feaf5ad24eb85657a65733f4ebe275bc6ecf93113201c2585896bbdf3520f6a9" exitCode=0
Nov 04 03:49:58.041177 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.040825    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" event={"ID":"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e","Type":"ContainerDied","Data":"feaf5ad24eb85657a65733f4ebe275bc6ecf93113201c2585896bbdf3520f6a9"}
Nov 04 03:49:58.041177 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.040927    4420 scope.go:122] "RemoveContainer" containerID="b114c09d6992206d59be7de814dba1a468fb8634556a64da10f048e20d2e27e1"
Nov 04 03:49:58.153352 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.153329    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:49:58.241739 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.241714    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-kube-api-access-g6xpm\" (UniqueName: \"kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-kube-api-access-g6xpm\") pod \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") "
Nov 04 03:49:58.241902 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.241752    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-envoy\") pod \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") "
Nov 04 03:49:58.241902 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.241786    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-certs\") pod \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") "
Nov 04 03:49:58.241902 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.241826    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-token\" (UniqueName: \"kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-token\") pod \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") "
Nov 04 03:49:58.241902 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.241879    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-credential-socket\") pod \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") "
Nov 04 03:49:58.242136 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.241926    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istiod-ca-cert\") pod \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") "
Nov 04 03:49:58.242136 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.241951    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-podinfo\") pod \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") "
Nov 04 03:49:58.242136 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.241998    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-data\") pod \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") "
Nov 04 03:49:58.242136 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.242069    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-socket\") pod \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\" (UID: \"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e\") "
Nov 04 03:49:58.242136 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.242099    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-certs" pod "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" (UID: "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:49:58.242401 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.242237    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-credential-socket" pod "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" (UID: "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:49:58.242401 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.242317    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-data" pod "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" (UID: "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:49:58.242513 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.242432    4420 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-data\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:58.242513 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.242454    4420 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:58.242513 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.242471    4420 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-credential-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:58.242513 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.242475    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-socket" pod "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" (UID: "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:49:58.242732 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.242518    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istiod-ca-cert" pod "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" (UID: "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:49:58.244076 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.244047    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-kube-api-access-g6xpm" pod "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" (UID: "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e"). InnerVolumeSpecName "kube-api-access-g6xpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:58.244076 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.244060    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-envoy" pod "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" (UID: "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:49:58.244076 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.244078    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-token" pod "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" (UID: "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:49:58.244305 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.244141    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-podinfo" pod "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" (UID: "0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 03:49:58.343149 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.343123    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6xpm\" (UniqueName: \"kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-kube-api-access-g6xpm\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:58.343149 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.343151    4420 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-envoy\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:58.343294 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.343164    4420 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-token\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:58.343294 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.343177    4420 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istiod-ca-cert\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:58.343294 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.343187    4420 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-istio-podinfo\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:58.343294 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.343197    4420 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e-workload-socket\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:49:58.349431 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.349405    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"]
Nov 04 03:49:58.366927 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.366900    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"]
Nov 04 03:49:58.376203 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.376180    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"]
Nov 04 03:49:58.470432 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.470399    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:58.898032 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:58.898004    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:59.053113 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:59.053082    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"
Nov 04 03:49:59.053113 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:59.053091    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5" event={"ID":"0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e","Type":"ContainerDied","Data":"9b7d4da662870392d3b446d529883cd99e068be61d1c636b4950cc3f1a09e436"}
Nov 04 03:49:59.053650 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:59.053165    4420 scope.go:122] "RemoveContainer" containerID="feaf5ad24eb85657a65733f4ebe275bc6ecf93113201c2585896bbdf3520f6a9"
Nov 04 03:49:59.209623 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:59.209553    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"]
Nov 04 03:49:59.256137 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:49:59.256114    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-951893-openshift-default-586cb5d9bf-xmfj5"]
Nov 04 03:50:00.305690 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:00.305657    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" path="/var/lib/kubelet/pods/0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e/volumes"
Nov 04 03:50:00.429957 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:00.429924    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:00.649767 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:00.649684    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:02.649184 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:50:02.649142    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:50:03.085893 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:03.085859    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:50:03.758456 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:03.758417    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:07.157474 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:07.157439    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:07.357082 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:07.357042    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:07.505534 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:07.505498    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:10.768072 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:10.768042    4420 scope.go:122] "RemoveContainer" containerID="707f5b5ad54487e09c22973d88cea216bd2ed3442b5f9efe13c5ff25d0cbc2fb"
Nov 04 03:50:11.205867 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:11.205761    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:13.369204 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.369160    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t" podUID="178304db-3ebd-4993-abd1-b283b7274d8a" containerName="main" containerID="cri-o://e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605" gracePeriod=45
Nov 04 03:50:13.389307 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.389257    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf" podUID="70953f5d-4cbc-4cb5-8136-4045ee3af987" containerName="main" containerID="cri-o://81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f" gracePeriod=45
Nov 04 03:50:13.394750 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.394720    4420 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf" podUID="3ce57bc5-3ee7-48f5-8cf3-6b29379219fc" containerName="main" containerID="cri-o://06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538" gracePeriod=45
Nov 04 03:50:13.428637 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.428606    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:13.648285 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.648262    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:50:13.667101 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.667077    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:50:13.676431 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.676412    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:50:13.783884 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.783834    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-tmp-dir\") pod \"70953f5d-4cbc-4cb5-8136-4045ee3af987\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") "
Nov 04 03:50:13.784085 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.783914    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tls-certs\" (UniqueName: \"kubernetes.io/secret/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tls-certs\") pod \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") "
Nov 04 03:50:13.784085 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.783951    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70953f5d-4cbc-4cb5-8136-4045ee3af987-tls-certs\" (UniqueName: \"kubernetes.io/secret/70953f5d-4cbc-4cb5-8136-4045ee3af987-tls-certs\") pod \"70953f5d-4cbc-4cb5-8136-4045ee3af987\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") "
Nov 04 03:50:13.784085 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784012    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-home\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-home\") pod \"70953f5d-4cbc-4cb5-8136-4045ee3af987\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") "
Nov 04 03:50:13.784085 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784042    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/178304db-3ebd-4993-abd1-b283b7274d8a-kube-api-access-r9bd2\" (UniqueName: \"kubernetes.io/projected/178304db-3ebd-4993-abd1-b283b7274d8a-kube-api-access-r9bd2\") pod \"178304db-3ebd-4993-abd1-b283b7274d8a\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") "
Nov 04 03:50:13.784085 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784071    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-dshm\") pod \"178304db-3ebd-4993-abd1-b283b7274d8a\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") "
Nov 04 03:50:13.784351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784125    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tmp-dir\") pod \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") "
Nov 04 03:50:13.784351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784138    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-tmp-dir" pod "70953f5d-4cbc-4cb5-8136-4045ee3af987" (UID: "70953f5d-4cbc-4cb5-8136-4045ee3af987"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.784351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784151    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-home\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-home\") pod \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") "
Nov 04 03:50:13.784351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784187    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/178304db-3ebd-4993-abd1-b283b7274d8a-tls-certs\" (UniqueName: \"kubernetes.io/secret/178304db-3ebd-4993-abd1-b283b7274d8a-tls-certs\") pod \"178304db-3ebd-4993-abd1-b283b7274d8a\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") "
Nov 04 03:50:13.784351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784215    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-model-cache\") pod \"178304db-3ebd-4993-abd1-b283b7274d8a\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") "
Nov 04 03:50:13.784351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784259    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70953f5d-4cbc-4cb5-8136-4045ee3af987-kube-api-access-v2c2z\" (UniqueName: \"kubernetes.io/projected/70953f5d-4cbc-4cb5-8136-4045ee3af987-kube-api-access-v2c2z\") pod \"70953f5d-4cbc-4cb5-8136-4045ee3af987\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") "
Nov 04 03:50:13.784351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784288    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-tmp-dir\") pod \"178304db-3ebd-4993-abd1-b283b7274d8a\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") "
Nov 04 03:50:13.784351 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784314    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-model-cache\") pod \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") "
Nov 04 03:50:13.784749 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784367    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-home\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-home\") pod \"178304db-3ebd-4993-abd1-b283b7274d8a\" (UID: \"178304db-3ebd-4993-abd1-b283b7274d8a\") "
Nov 04 03:50:13.784749 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784395    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-dshm\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-dshm\") pod \"70953f5d-4cbc-4cb5-8136-4045ee3af987\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") "
Nov 04 03:50:13.784749 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784422    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-model-cache\") pod \"70953f5d-4cbc-4cb5-8136-4045ee3af987\" (UID: \"70953f5d-4cbc-4cb5-8136-4045ee3af987\") "
Nov 04 03:50:13.784749 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784456    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-dshm\") pod \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") "
Nov 04 03:50:13.784749 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784511    4420 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-kube-api-access-ljmf2\" (UniqueName: \"kubernetes.io/projected/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-kube-api-access-ljmf2\") pod \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\" (UID: \"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc\") "
Nov 04 03:50:13.785025 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784771    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tmp-dir" pod "3ce57bc5-3ee7-48f5-8cf3-6b29379219fc" (UID: "3ce57bc5-3ee7-48f5-8cf3-6b29379219fc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.785025 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.784836    4420 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-tmp-dir\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.785133 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.785086    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-home" pod "3ce57bc5-3ee7-48f5-8cf3-6b29379219fc" (UID: "3ce57bc5-3ee7-48f5-8cf3-6b29379219fc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.786102 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.785352    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-model-cache" pod "70953f5d-4cbc-4cb5-8136-4045ee3af987" (UID: "70953f5d-4cbc-4cb5-8136-4045ee3af987"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.786102 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.785581    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-tmp-dir" pod "178304db-3ebd-4993-abd1-b283b7274d8a" (UID: "178304db-3ebd-4993-abd1-b283b7274d8a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.786102 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.785628    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-model-cache" pod "3ce57bc5-3ee7-48f5-8cf3-6b29379219fc" (UID: "3ce57bc5-3ee7-48f5-8cf3-6b29379219fc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.786102 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.785797    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-home" pod "178304db-3ebd-4993-abd1-b283b7274d8a" (UID: "178304db-3ebd-4993-abd1-b283b7274d8a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.786415 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.786265    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-home" pod "70953f5d-4cbc-4cb5-8136-4045ee3af987" (UID: "70953f5d-4cbc-4cb5-8136-4045ee3af987"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.786552 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.786519    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-model-cache" pod "178304db-3ebd-4993-abd1-b283b7274d8a" (UID: "178304db-3ebd-4993-abd1-b283b7274d8a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.787731 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.787693    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/178304db-3ebd-4993-abd1-b283b7274d8a-kube-api-access-r9bd2" pod "178304db-3ebd-4993-abd1-b283b7274d8a" (UID: "178304db-3ebd-4993-abd1-b283b7274d8a"). InnerVolumeSpecName "kube-api-access-r9bd2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:13.788810 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.788775    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70953f5d-4cbc-4cb5-8136-4045ee3af987-tls-certs" pod "70953f5d-4cbc-4cb5-8136-4045ee3af987" (UID: "70953f5d-4cbc-4cb5-8136-4045ee3af987"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:13.788942 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.788810    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tls-certs" pod "3ce57bc5-3ee7-48f5-8cf3-6b29379219fc" (UID: "3ce57bc5-3ee7-48f5-8cf3-6b29379219fc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:13.789812 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.789786    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-dshm" pod "178304db-3ebd-4993-abd1-b283b7274d8a" (UID: "178304db-3ebd-4993-abd1-b283b7274d8a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.789812 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.789798    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-dshm" pod "70953f5d-4cbc-4cb5-8136-4045ee3af987" (UID: "70953f5d-4cbc-4cb5-8136-4045ee3af987"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.790034 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.789786    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/178304db-3ebd-4993-abd1-b283b7274d8a-tls-certs" pod "178304db-3ebd-4993-abd1-b283b7274d8a" (UID: "178304db-3ebd-4993-abd1-b283b7274d8a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:13.790034 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.789962    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-kube-api-access-ljmf2" pod "3ce57bc5-3ee7-48f5-8cf3-6b29379219fc" (UID: "3ce57bc5-3ee7-48f5-8cf3-6b29379219fc"). InnerVolumeSpecName "kube-api-access-ljmf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:13.790340 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.790320    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-dshm" pod "3ce57bc5-3ee7-48f5-8cf3-6b29379219fc" (UID: "3ce57bc5-3ee7-48f5-8cf3-6b29379219fc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:13.791319 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.791304    4420 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70953f5d-4cbc-4cb5-8136-4045ee3af987-kube-api-access-v2c2z" pod "70953f5d-4cbc-4cb5-8136-4045ee3af987" (UID: "70953f5d-4cbc-4cb5-8136-4045ee3af987"). InnerVolumeSpecName "kube-api-access-v2c2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:13.885696 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885614    4420 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-home\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885696 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885650    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9bd2\" (UniqueName: \"kubernetes.io/projected/178304db-3ebd-4993-abd1-b283b7274d8a-kube-api-access-r9bd2\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885696 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885665    4420 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-dshm\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885696 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885677    4420 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tmp-dir\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885696 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885689    4420 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-home\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885701    4420 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/178304db-3ebd-4993-abd1-b283b7274d8a-tls-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885715    4420 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-model-cache\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885728    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2c2z\" (UniqueName: \"kubernetes.io/projected/70953f5d-4cbc-4cb5-8136-4045ee3af987-kube-api-access-v2c2z\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885742    4420 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-tmp-dir\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885754    4420 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-model-cache\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885765    4420 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/178304db-3ebd-4993-abd1-b283b7274d8a-home\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885776    4420 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-dshm\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885787    4420 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70953f5d-4cbc-4cb5-8136-4045ee3af987-model-cache\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885798    4420 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-dshm\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885810    4420 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljmf2\" (UniqueName: \"kubernetes.io/projected/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-kube-api-access-ljmf2\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885821    4420 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc-tls-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:13.885992 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:13.885835    4420 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70953f5d-4cbc-4cb5-8136-4045ee3af987-tls-certs\") on node \"ip-10-0-133-201.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:14.121623 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.121585    4420 generic.go:356] "Generic (PLEG): container finished" podID="70953f5d-4cbc-4cb5-8136-4045ee3af987" containerID="81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f" exitCode=0
Nov 04 03:50:14.121797 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.121640    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"
Nov 04 03:50:14.121797 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.121668    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf" event={"ID":"70953f5d-4cbc-4cb5-8136-4045ee3af987","Type":"ContainerDied","Data":"81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f"}
Nov 04 03:50:14.121797 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.121709    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf" event={"ID":"70953f5d-4cbc-4cb5-8136-4045ee3af987","Type":"ContainerDied","Data":"c5e32ee05480ae796c0149683c0bbccf39dd9db76fb0053755d8de128f96ef56"}
Nov 04 03:50:14.121797 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.121726    4420 scope.go:122] "RemoveContainer" containerID="81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f"
Nov 04 03:50:14.122956 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.122938    4420 generic.go:356] "Generic (PLEG): container finished" podID="3ce57bc5-3ee7-48f5-8cf3-6b29379219fc" containerID="06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538" exitCode=0
Nov 04 03:50:14.123065 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.122969    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf" event={"ID":"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc","Type":"ContainerDied","Data":"06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538"}
Nov 04 03:50:14.123065 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.122998    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"
Nov 04 03:50:14.123065 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.123006    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf" event={"ID":"3ce57bc5-3ee7-48f5-8cf3-6b29379219fc","Type":"ContainerDied","Data":"7901701b5e904b122f467017da6331ca1aba23474bd8076de7908a0ebe5ffd15"}
Nov 04 03:50:14.124529 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.124505    4420 generic.go:356] "Generic (PLEG): container finished" podID="178304db-3ebd-4993-abd1-b283b7274d8a" containerID="e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605" exitCode=0
Nov 04 03:50:14.124642 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.124533    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t" event={"ID":"178304db-3ebd-4993-abd1-b283b7274d8a","Type":"ContainerDied","Data":"e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605"}
Nov 04 03:50:14.124642 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.124554    4420 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"
Nov 04 03:50:14.124642 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.124571    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t" event={"ID":"178304db-3ebd-4993-abd1-b283b7274d8a","Type":"ContainerDied","Data":"e4c2ae9cc96600f9584a10f7382b17461b4efd9a06947ac685b49b0149478f08"}
Nov 04 03:50:14.130701 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.130678    4420 scope.go:122] "RemoveContainer" containerID="81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f"
Nov 04 03:50:14.131049 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:50:14.131028    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f\": container with ID starting with 81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f not found: ID does not exist" containerID="81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f"
Nov 04 03:50:14.131121 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.131054    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f"} err="failed to get container status \"81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f\": rpc error: code = NotFound desc = could not find container \"81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f\": container with ID starting with 81936f73810e3c347d9b2fd2769a90dda75bd466afa9dd8ad95a437cf3134d0f not found: ID does not exist"
Nov 04 03:50:14.131121 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.131072    4420 scope.go:122] "RemoveContainer" containerID="06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538"
Nov 04 03:50:14.139308 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.139290    4420 scope.go:122] "RemoveContainer" containerID="06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538"
Nov 04 03:50:14.139546 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:50:14.139525    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538\": container with ID starting with 06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538 not found: ID does not exist" containerID="06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538"
Nov 04 03:50:14.139593 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.139551    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538"} err="failed to get container status \"06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538\": rpc error: code = NotFound desc = could not find container \"06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538\": container with ID starting with 06403d874041188ccbebfef2d8bae1003dd204e623eb618cb8636c21895b6538 not found: ID does not exist"
Nov 04 03:50:14.139593 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.139566    4420 scope.go:122] "RemoveContainer" containerID="e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605"
Nov 04 03:50:14.146516 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.146499    4420 scope.go:122] "RemoveContainer" containerID="e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605"
Nov 04 03:50:14.146790 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:50:14.146772    4420 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605\": container with ID starting with e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605 not found: ID does not exist" containerID="e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605"
Nov 04 03:50:14.146959 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.146794    4420 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605"} err="failed to get container status \"e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605\": rpc error: code = NotFound desc = could not find container \"e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605\": container with ID starting with e3572a47ed78c47b159834020e654706b87e1c9cf4534d974d476cc6e4d07605 not found: ID does not exist"
Nov 04 03:50:14.275824 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.275794    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"]
Nov 04 03:50:14.286324 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.286291    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/distinct-2-w2-fba151-kserve-78cd64fbd-pzp8t"]
Nov 04 03:50:14.304967 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.304945    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="178304db-3ebd-4993-abd1-b283b7274d8a" path="/var/lib/kubelet/pods/178304db-3ebd-4993-abd1-b283b7274d8a/volumes"
Nov 04 03:50:14.388884 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.388802    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"]
Nov 04 03:50:14.395595 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.395574    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf"]
Nov 04 03:50:14.497807 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.497777    4420 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"]
Nov 04 03:50:14.502166 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.502144    4420 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-fba151/simulator-w2-fba151-kserve-7445bbc99c-dg7bf"]
Nov 04 03:50:14.638407 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:14.638357    4420 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-fba151/premium-w2-fba151-kserve-575dfdb84d-l5dcf" podUID="70953f5d-4cbc-4cb5-8136-4045ee3af987" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:50:16.304884 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:16.304829    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ce57bc5-3ee7-48f5-8cf3-6b29379219fc" path="/var/lib/kubelet/pods/3ce57bc5-3ee7-48f5-8cf3-6b29379219fc/volumes"
Nov 04 03:50:16.305260 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:16.305216    4420 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70953f5d-4cbc-4cb5-8136-4045ee3af987" path="/var/lib/kubelet/pods/70953f5d-4cbc-4cb5-8136-4045ee3af987/volumes"
Nov 04 03:50:18.280375 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:18.280334    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:19.403673 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:19.403644    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:25.097774 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:25.097738    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rqz2j"]
Nov 04 03:50:25.397652 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:25.397565    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:25.544587 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:25.544542    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:32.647211 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:50:32.647178    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:50:36.194275 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:36.194230    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:36.394598 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:36.394552    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:36.582738 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:36.582704    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:40.064194 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:40.064151    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:40.560623 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:40.560590    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:42.994171 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:42.994126    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:43.316266 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:43.316175    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:56.038114 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:50:56.038042    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-rqz2j_5805a14d-7dad-43be-9437-292b8982a631/limitador/0.log"
Nov 04 03:51:02.648731 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:51:02.648695    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:51:06.277478 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:06.277440    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-jtvzt_ce464396-b997-492c-865c-5def7527f977/manager/0.log"
Nov 04 03:51:07.778262 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:07.778226    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rcb44_2fe2560c-8be3-41e7-a487-70c9075c72ba/manager/0.log"
Nov 04 03:51:08.410029 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:08.409999    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-fbbwk_7d26d201-c93f-479b-8975-dacdbfe056e2/manager/0.log"
Nov 04 03:51:09.427077 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:09.427046    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-tkm8m_fa9c93bf-935f-4283-8304-d42e51ed5e4c/server/0.log"
Nov 04 03:51:11.497246 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:11.497206    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:51:32.649113 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:51:32.649078    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:51:46.637372 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:46.637332    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rcb44_2fe2560c-8be3-41e7-a487-70c9075c72ba/manager/0.log"
Nov 04 03:51:47.281187 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:47.281159    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-fbbwk_7d26d201-c93f-479b-8975-dacdbfe056e2/manager/0.log"
Nov 04 03:51:48.226346 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:48.226316    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-tkm8m_fa9c93bf-935f-4283-8304-d42e51ed5e4c/server/0.log"
Nov 04 03:51:55.155009 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:55.154977    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-rqz2j_5805a14d-7dad-43be-9437-292b8982a631/limitador/0.log"
Nov 04 03:51:55.387775 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:55.387745    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-jtvzt_ce464396-b997-492c-865c-5def7527f977/manager/0.log"
Nov 04 03:51:57.302408 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:57.302375    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7659c7d778-lxgpd_71822c20-2453-4e37-96a9-a0f20d9c11b8/kube-auth-proxy/0.log"
Nov 04 03:51:58.034632 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:58.034599    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-fd2nl_dae825f6-0aa1-45e3-bba4-c2c83d6d9366/istio-proxy/0.log"
Nov 04 03:51:58.824145 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:58.824105    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-j6k8d_094e67aa-800c-4f21-826f-2240183d969e/istio-proxy/0.log"
Nov 04 03:51:59.058987 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:51:59.058958    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-jx8n4_3e18db8b-6a81-49e5-9524-1b65624cac59/istio-proxy/0.log"
Nov 04 03:52:02.652165 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:52:02.652127    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:52:06.115734 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:06.115702    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:10.515153 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:10.515127    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:52:10.515635 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:10.515336    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:52:11.865569 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:11.865534    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:13.552235 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:13.552200    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:13.763233 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:13.763201    4420 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:24.860123 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:24.860092    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tmrtb_39429a51-69a0-4bce-9029-773d5aad2a87/global-pull-secret-syncer/0.log"
Nov 04 03:52:25.382343 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:25.382313    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rcb44_2fe2560c-8be3-41e7-a487-70c9075c72ba/manager/0.log"
Nov 04 03:52:26.010946 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:26.010909    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-fbbwk_7d26d201-c93f-479b-8975-dacdbfe056e2/manager/0.log"
Nov 04 03:52:26.580700 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:26.580669    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-tkm8m_fa9c93bf-935f-4283-8304-d42e51ed5e4c/server/0.log"
Nov 04 03:52:27.057957 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:27.057922    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6x62l_3aef6eaf-2dde-4706-bcf9-77bf9c447028/konnectivity-agent/0.log"
Nov 04 03:52:28.839878 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:28.839831    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-201.us-west-2.compute.internal_72ddb4465d8ebfdcb6c7be795263771f/haproxy/0.log"
Nov 04 03:52:30.678414 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:30.678388    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rcb44_2fe2560c-8be3-41e7-a487-70c9075c72ba/manager/0.log"
Nov 04 03:52:31.331739 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:31.331708    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-fbbwk_7d26d201-c93f-479b-8975-dacdbfe056e2/manager/0.log"
Nov 04 03:52:31.788700 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:31.788669    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-tkm8m_fa9c93bf-935f-4283-8304-d42e51ed5e4c/server/0.log"
Nov 04 03:52:32.647283 ip-10-0-133-201 kubenswrapper[4420]: E1104 03:52:32.647248    4420 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-vtl99" containerName="extractor"
Nov 04 03:52:33.702374 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702341    4420 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"]
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702669    4420 cpu_manager.go:414] "containerMap: removing container" podUID="70953f5d-4cbc-4cb5-8136-4045ee3af987" containerName="main"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702679    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70953f5d-4cbc-4cb5-8136-4045ee3af987" containerName="main"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702691    4420 cpu_manager.go:414] "containerMap: removing container" podUID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9" containerName="cleanup"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702696    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9" containerName="cleanup"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702704    4420 cpu_manager.go:414] "containerMap: removing container" podUID="3ce57bc5-3ee7-48f5-8cf3-6b29379219fc" containerName="main"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702709    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ce57bc5-3ee7-48f5-8cf3-6b29379219fc" containerName="main"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702715    4420 cpu_manager.go:414] "containerMap: removing container" podUID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerName="istio-proxy"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702720    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerName="istio-proxy"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702733    4420 cpu_manager.go:414] "containerMap: removing container" podUID="856ee09f-b9e3-46d2-9d2c-c49ea8000274" containerName="cleanup"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702738    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="856ee09f-b9e3-46d2-9d2c-c49ea8000274" containerName="cleanup"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702744    4420 cpu_manager.go:414] "containerMap: removing container" podUID="4dd3b4c8-659f-49b2-99f8-13355a4ede22" containerName="payload-processing"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702748    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4dd3b4c8-659f-49b2-99f8-13355a4ede22" containerName="payload-processing"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702759    4420 cpu_manager.go:414] "containerMap: removing container" podUID="9aa6de23-5c26-4d8d-acc8-03f6963b1dcd" containerName="payload-processing"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702763    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9aa6de23-5c26-4d8d-acc8-03f6963b1dcd" containerName="payload-processing"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702774    4420 cpu_manager.go:414] "containerMap: removing container" podUID="178304db-3ebd-4993-abd1-b283b7274d8a" containerName="main"
Nov 04 03:52:33.702769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702778    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="178304db-3ebd-4993-abd1-b283b7274d8a" containerName="main"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702790    4420 cpu_manager.go:414] "containerMap: removing container" podUID="856ee09f-b9e3-46d2-9d2c-c49ea8000274" containerName="cleanup"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702794    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="856ee09f-b9e3-46d2-9d2c-c49ea8000274" containerName="cleanup"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702805    4420 cpu_manager.go:414] "containerMap: removing container" podUID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9" containerName="cleanup"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702809    4420 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9" containerName="cleanup"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702871    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="178304db-3ebd-4993-abd1-b283b7274d8a" containerName="main"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702880    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="856ee09f-b9e3-46d2-9d2c-c49ea8000274" containerName="cleanup"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702886    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a9287c7-4a73-4ec5-9fa3-b22e78c8b25e" containerName="istio-proxy"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702893    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="856ee09f-b9e3-46d2-9d2c-c49ea8000274" containerName="cleanup"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702899    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9" containerName="cleanup"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702904    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9" containerName="cleanup"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702909    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="70953f5d-4cbc-4cb5-8136-4045ee3af987" containerName="main"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702915    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0098b4f-d0d9-4de0-b10c-871e60a48ef9" containerName="cleanup"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702920    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ce57bc5-3ee7-48f5-8cf3-6b29379219fc" containerName="main"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702926    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="4dd3b4c8-659f-49b2-99f8-13355a4ede22" containerName="payload-processing"
Nov 04 03:52:33.703288 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.702933    4420 memory_manager.go:365] "RemoveStaleState removing state" podUID="9aa6de23-5c26-4d8d-acc8-03f6963b1dcd" containerName="payload-processing"
Nov 04 03:52:33.706141 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.706120    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.761373 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.761341    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"]
Nov 04 03:52:33.791960 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.791933    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e8a234fd-4758-4240-a6b6-b67c9145ead1-proc\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.792252 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.791989    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8a234fd-4758-4240-a6b6-b67c9145ead1-lib-modules\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.792252 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.792084    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e8a234fd-4758-4240-a6b6-b67c9145ead1-podres\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.792252 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.792138    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8a234fd-4758-4240-a6b6-b67c9145ead1-sys\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.792252 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.792185    4420 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7gsq\" (UniqueName: \"kubernetes.io/projected/e8a234fd-4758-4240-a6b6-b67c9145ead1-kube-api-access-n7gsq\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.802990 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.802972    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x26lf\"/\"openshift-service-ca.crt\""
Nov 04 03:52:33.803107 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.803090    4420 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x26lf\"/\"kube-root-ca.crt\""
Nov 04 03:52:33.803151 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.803141    4420 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x26lf\"/\"default-dockercfg-l6db9\""
Nov 04 03:52:33.893574 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.893534    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8a234fd-4758-4240-a6b6-b67c9145ead1-lib-modules\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.893768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.893586    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e8a234fd-4758-4240-a6b6-b67c9145ead1-podres\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.893768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.893620    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8a234fd-4758-4240-a6b6-b67c9145ead1-sys\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.893768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.893640    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7gsq\" (UniqueName: \"kubernetes.io/projected/e8a234fd-4758-4240-a6b6-b67c9145ead1-kube-api-access-n7gsq\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.893768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.893674    4420 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e8a234fd-4758-4240-a6b6-b67c9145ead1-proc\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.893768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.893676    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8a234fd-4758-4240-a6b6-b67c9145ead1-lib-modules\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.893768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.893735    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e8a234fd-4758-4240-a6b6-b67c9145ead1-podres\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.893768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.893749    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e8a234fd-4758-4240-a6b6-b67c9145ead1-proc\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.893768 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.893735    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8a234fd-4758-4240-a6b6-b67c9145ead1-sys\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:33.952237 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:33.952206    4420 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7gsq\" (UniqueName: \"kubernetes.io/projected/e8a234fd-4758-4240-a6b6-b67c9145ead1-kube-api-access-n7gsq\") pod \"perf-node-gather-daemonset-d9sfh\" (UID: \"e8a234fd-4758-4240-a6b6-b67c9145ead1\") " pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:34.018006 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:34.017978    4420 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:34.386769 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:34.386685    4420 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"]
Nov 04 03:52:34.393124 ip-10-0-133-201 kubenswrapper[4420]: W1104 03:52:34.393094    4420 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode8a234fd_4758_4240_a6b6_b67c9145ead1.slice/crio-783879f5bd4b947a39c1dae9ca01d3d97aaaf4e4c5883cd9d1eedc0eb0481cbb WatchSource:0}: Error finding container 783879f5bd4b947a39c1dae9ca01d3d97aaaf4e4c5883cd9d1eedc0eb0481cbb: Status 404 returned error can't find the container with id 783879f5bd4b947a39c1dae9ca01d3d97aaaf4e4c5883cd9d1eedc0eb0481cbb
Nov 04 03:52:34.731727 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:34.731633    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh" event={"ID":"e8a234fd-4758-4240-a6b6-b67c9145ead1","Type":"ContainerStarted","Data":"3ece2279921ee934d74672840f860e327f4349a3bc2ff4febacd5a9ca2ae8744"}
Nov 04 03:52:34.731727 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:34.731692    4420 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh" event={"ID":"e8a234fd-4758-4240-a6b6-b67c9145ead1","Type":"ContainerStarted","Data":"783879f5bd4b947a39c1dae9ca01d3d97aaaf4e4c5883cd9d1eedc0eb0481cbb"}
Nov 04 03:52:34.732262 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:34.731772    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:34.841500 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:34.841448    4420 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh" podStartSLOduration=1.841433992 podStartE2EDuration="1.841433992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:52:34.840020753 +0000 UTC m=+2124.654812042" watchObservedRunningTime="2026-11-04 03:52:34.841433992 +0000 UTC m=+2124.656225269"
Nov 04 03:52:35.929919 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:35.929876    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-rqz2j_5805a14d-7dad-43be-9437-292b8982a631/limitador/0.log"
Nov 04 03:52:35.990125 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:35.990092    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rcb44_2fe2560c-8be3-41e7-a487-70c9075c72ba/manager/0.log"
Nov 04 03:52:36.398165 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:36.398133    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-jtvzt_ce464396-b997-492c-865c-5def7527f977/manager/0.log"
Nov 04 03:52:36.619196 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:36.619159    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-fbbwk_7d26d201-c93f-479b-8975-dacdbfe056e2/manager/0.log"
Nov 04 03:52:36.961223 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:36.961195    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-tkm8m_fa9c93bf-935f-4283-8304-d42e51ed5e4c/server/0.log"
Nov 04 03:52:40.746492 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:40.746466    4420 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-x26lf/perf-node-gather-daemonset-d9sfh"
Nov 04 03:52:42.778206 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:42.778175    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-d8tbq_5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c/kube-state-metrics/0.log"
Nov 04 03:52:43.203497 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:43.203412    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-d8tbq_5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c/kube-rbac-proxy-main/0.log"
Nov 04 03:52:43.252792 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:43.252767    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7659c7d778-lxgpd_71822c20-2453-4e37-96a9-a0f20d9c11b8/kube-auth-proxy/0.log"
Nov 04 03:52:43.640931 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:43.640898    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-d8tbq_5bd4d229-f69b-4f92-9f92-5f6fe1e3b42c/kube-rbac-proxy-self/0.log"
Nov 04 03:52:43.938821 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:43.938742    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-fd2nl_dae825f6-0aa1-45e3-bba4-c2c83d6d9366/istio-proxy/0.log"
Nov 04 03:52:44.530784 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:44.530759    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-mrzfr_082d366a-f1ca-4eba-a55d-d1b683942520/monitoring-plugin/0.log"
Nov 04 03:52:44.624497 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:44.624468    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-j6k8d_094e67aa-800c-4f21-826f-2240183d969e/istio-proxy/0.log"
Nov 04 03:52:44.845300 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:44.845219    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-jx8n4_3e18db8b-6a81-49e5-9524-1b65624cac59/istio-proxy/0.log"
Nov 04 03:52:44.970076 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:44.970052    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c25dk_91ee7662-60fe-42cf-849a-0d166cb7f5e1/node-exporter/0.log"
Nov 04 03:52:45.419744 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:45.419713    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c25dk_91ee7662-60fe-42cf-849a-0d166cb7f5e1/kube-rbac-proxy/0.log"
Nov 04 03:52:45.857096 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:45.857067    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c25dk_91ee7662-60fe-42cf-849a-0d166cb7f5e1/init-textfile/0.log"
Nov 04 03:52:49.015695 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:49.015662    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-g8bvk_f727f221-d129-42c3-9b3c-c44a07dff2a9/kube-rbac-proxy-main/0.log"
Nov 04 03:52:49.449044 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:49.448963    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-g8bvk_f727f221-d129-42c3-9b3c-c44a07dff2a9/kube-rbac-proxy-self/0.log"
Nov 04 03:52:49.885523 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:49.885491    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-g8bvk_f727f221-d129-42c3-9b3c-c44a07dff2a9/openshift-state-metrics/0.log"
Nov 04 03:52:50.329826 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:50.329784    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/0.log"
Nov 04 03:52:50.331175 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:50.331145    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/prometheus/1.log"
Nov 04 03:52:50.658358 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:50.658277    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/config-reloader/0.log"
Nov 04 03:52:51.098175 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:51.098144    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/thanos-sidecar/0.log"
Nov 04 03:52:51.548027 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:51.547997    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/kube-rbac-proxy-web/0.log"
Nov 04 03:52:51.806897 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:51.806793    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-rqz2j_5805a14d-7dad-43be-9437-292b8982a631/limitador/0.log"
Nov 04 03:52:51.992607 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:51.992575    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/kube-rbac-proxy/0.log"
Nov 04 03:52:52.071740 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:52.071659    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-jtvzt_ce464396-b997-492c-865c-5def7527f977/manager/0.log"
Nov 04 03:52:52.427489 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:52.427407    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/kube-rbac-proxy-thanos/0.log"
Nov 04 03:52:52.867627 ip-10-0-133-201 kubenswrapper[4420]: I1104 03:52:52.867592    4420 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_21860859-c6cc-4399-80db-f7da70e90cc3/init-config-reloader/0.log"